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Preface 


Hahe Muth dich deines eigenen Verstandes zu hedienen. 

(Have the courage to think for yourself.) 

Immanuel Kant, in Beantwortung der Frage: Was ist Aufkldrung? 


The disciplines of statistical mechanics and thermodynamics are very closely related, 
although their historical roots are separate. The founders of thermodynamics devel¬ 
oped their theories without the advantage of contemporary understanding of the 
atomic structure of matter. Statistical mechanics, which is built on this understanding, 
makes predictions of system behavior that lead to thermodynamic rules. In other 
words, statistical mechanics is a conceptual precursor to thermodynamics, although it 
is an historical latecomer. 

Unfortunately, despite their theoretical connection, statistical mechanics and ther¬ 
modynamics are often taught as separate fields of study. Even worse, thermodynamics 
is usually taught first, for the dubious reason that it is older than statistical mechanics. 
All too often the result is that students regard thermodynamics as a set of highly 
abstract mathematical relationships, the significance of which is not clear. 

This book is an effort to rectify the situation. It presents the two complementary 
aspects of thermal physics as a coherent theory of the properties of matter. My 
intention is that after working through this text a student will have solid foundations 
in both statistical mechanics and thermodynamics that will provide direct access to 
modern research. 


Guiding Principles 

In writing this book I have been guided by a number of principles, only some of which 
are shared by other textbooks in statistical mechanics and thermodynamics. 

• I have written this book for students, not professors. Many things that experts 
might take for granted are explained explicitly. Indeed, student contributions have 
been essential in constructing clear explanations that do not leave out ‘obvious’ 
steps that can be puzzling to someone new to this material. 

• The goal of the book is to provide the student with conceptual understanding, and 
the problems are designed in the service of this goal. They are quite challenging, 
but the challenges are primarily conceptual rather than algebraic or computa¬ 
tional. 

• I believe that students should have the opportunity to program models themselves 
and observe how the models behave under different conditions. Therefore, the 
problems include extensive use of computation. 
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• The book is intended to be accessible to students at different levels of preparation. 
I do not make a distinction between teaching the material at the advanced 
undergraduate and graduate levels, and indeed, I have taught such a course many 
times using the same approach and much of the same material for both groups. As 
the mathematics is entirely self-contained, students can master all of the material 
even if their mathematical preparation has some gaps. Graduate students with 
previous courses on these topics should be able to use the book with self-study 
to make up for any gaps in their training. 

• After working through this text, a student should be well prepared to con¬ 
tinue with more specialized topics in thermodynamics, statistical mechanics, and 
condensed-matter physics. 


Pedagogical Principles 

The over-arching goals described above result in some unique features of my approach 
to the teaching of statistical mechanics and thermodynamics, which I think merit 
specific mention. 

Teaching Statistical Mechanics 

• The book begins with classical statistical mechanics to postpone the complica¬ 
tions of quantum measurement until the basic ideas are established. 

• I have defined ensembles in terms of probabilities, in keeping with Boltzmann’s 
vision. In particular, the discussion of statistical mechanics is based on Boltz¬ 
mann’s 1877 definition of entropy. This is not the definition usually found in 
textbooks, but what he actually wrote. The use of Boltzmann’s definition is one of 
the key features of the book that enables students to obtain a deep understanding 
of the foundations of both statistical mechanics and thermodynamics. 

• A self-contained discussion of probability theory is presented for both discrete 
and continuous random variables, including all material needed to understand 
basic statistical mechanics. This material would be superfluous if the physics 
curriculum were to include a course in probability theory, but unfortunately, that 
is not usually the case. (A course in statistics would also be very valuable for 
physics students—but that is another story.) 

• Dirac delta functions are used to formulate the theory of continuous random 
variables, as well as to simplify the derivations of densities of states. This is not 
the way mathematicians tend to introduce probability densities, but I believe that 
it is by far the most useful approach for scientists. 

• Entropy is presented as a logical consequence of applying probability theory to 
systems containing a large number of particles, instead of just an equation to be 
memorized. 

• The entropy of the classical ideal gas is derived in detail. This provides an 
explicit example of an entropy function that exhibits all the properties postulated 
in thermodynamics. The example is simple enough to give every detail of the 
derivation of thermodynamic properties from statistical mechanics. 
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• The book includes an explanation of Gibbs’ paradox—which is not really para¬ 
doxical when you begin with Boltzmann’s 1877 definition of the entropy. 

• The apparent contradiction between observed irreversibility and time-reversal- 
invariant equations of motion is explained. I believe that this fills an important 
gap in a student’s appreciation of how a description of macroscopic phenomena 
can arise from statistical principles. 


Teaching Thermodynamics 

• The four fundamental postulates of thermodynamics proposed by Callen have 
been reformulated. The result is a set of six thermodynamic postulates, sequenced 
so as to build conceptual understanding. 

• Jacobians are used to simplify the derivation of thermodynamic identities. 

• The thermodynamic limit is discussed, but the validity of thermodynamics and 
statistical mechanics does not rely on taking the limit of infinite size. This is 
important if thermodynamics is to be applied to real systems, but is sometimes 
neglected in textbooks. 

• My treatment includes thermodynamics of non-extensive systems. This allows 
me to include descriptions of systems with surfaces and systems enclosed in 
containers. 


Organization and Content 

The principles I have described above lead me to an organization for the book that 
is quite different from what has become the norm. As was stated above, while most 
texts on thermal physics begin with thermodynamics for historical reasons, I think it 
is far preferable from the perspective of pedagogy to begin with statistical mechanics, 
including an introduction to those parts of probability theory that are essential to 
statistical mechanics. 

To postpone the conceptual problems associated with quantum measurement, the 
initial discussion of statistical mechanics in Part I is limited to classical systems. 
The entropy of the classical ideal gas is derived in detail, with a clear justification 
for every step. A crucial aspect of the explanation and derivation of the entropy is 
the use of Boltzmann’s 1877 definition, which relates entropy to the probability of a 
macroscopic state. This definition provides a solid, intuitive understanding of what 
entropy is all about. It is my experience that after students have seen the derivation 
of the entropy of the classical ideal gas, they immediately understand the postulates 
of thermodynamics, since those postulates simply codify properties that they have 
derived explicitly for a special case. 

The treatment of statistical mechanics paves the way to the development of 
thermodynamics in Part II. While this development is largely based on the classic 
work by Herbert Callen (who was my thesis advisor), there are significant differences. 
Perhaps the most important is that I have relied entirely on Jacobians to derive 
thermodynamic identities. Instead of regarding such derivations with dread—as I did 
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when I first encountered them—my students tend to regard them as straightforward 
and rather easy. There are also several other changes in emphasis, such as a clarification 
of the postulates of thermodynamics and the inclusion of non-extensive systems; that 
is, finite systems that have surfaces or are enclosed in containers. 

Part III returns to classical statistical mechanics and develops the general theory 
directly, instead of using the common roundabout approach of taking the classical 
limit of quantum statistical mechanics. A chapter is devoted to a discussion of the 
apparent paradoxes between microscopic reversibility and macroscopic irreversibility. 

Part IV presents quantum statistical mechanics. The development begins by 
considering a probability distribution over all quantum states, instead of the common 
ad hoc restriction to eigenstates. In addition to the basic concepts, it covers black- 
body radiation, the harmonic crystal, and both Bose and Fermi gases. Because of 
their practical and theoretical importance, there is a separate chapter on insulators 
and semiconductors. The final chapter introduces the Ising model of magnetic phase 
transitions. 

The book contains about a hundred multi-part problems that should be considered 
as part of the text. In keeping with the level of the text, the problems are fairly 
challenging, and an effort has been made to avoid ‘plug and chug’ assignments. 
The challenges in the problems are mainly due to the probing of essential concepts, 
rather than mathematical complexities. A complete set of solutions to the problems 
is available from the publisher. 

Several of the problems, especially in the chapters on probability, rely on computer 
simulations to lead students to a deeper understanding. In the past I have suggested 
that my students use the C-|—h programming language, but for the last two years 
I have switched to VPython for its simplicity and the ease with which it generates 
graphs. An introduction to the basic features of VPython is given in in Appendix A. 
Most of my students have used VPython, but a significant fraction have chosen to use 
a different language—usually Java, C, or C-|—1-. I have not encountered any difficulties 
with allowing students to use the programming language of their choice. 


Two Semesters or One? 

The presentation of the material in this book is based primarily on a two-semester 
undergraduate course in thermal physics that I have taught several times at Carnegie 
Mellon University. Since two-semester undergraduate courses in thermal physics are 
rather unusual, its existence at Carnegie Mellon for several decades might be regarded 
as surprising. In my opinion, it should be the norm. Although it was quite reasonable 
to teach two semesters of classical mechanics and one semester of thermodynamics to 
undergraduates in the nineteenth century—the development of statistical mechanics 
was just beginning—it is not reasonable in the twenty-first century. 

However, even at Carnegie Mellon only the first semester of thermal physics is 
required. All physics majors take the first semester, and about half continue on to 
the second semester, accompanied by a few students from other departments. When I 
teach the course, the first semester covers the first two parts of the book (Chapters 1 
through 18), plus an overview of classical canonical ensembles (Chapter 18) and 
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quantum canonical ensembles (Chapter 22). This gives the students an introduction 
to statistical mechanics and a rather thorough knowledge of thermodynamics, even if 
they do not take the second semester. 

It is also possible to teach a one-semester course in thermal physics from this book 
using different choices of material. For example: 

• If the students have a strong background in probability theory (which is, unfortu¬ 
nately, fairly rare). Chapters 3 and 5 might be skipped to include more material 
in Parts III and IV. 

• If it is decided that students need a broader exposure to statistical mechanics, but 
that a less detailed study of thermodynamics is sufficient. Chapters 14 through 
17 could be skimmed to have time to study selected chapters in Parts III and IV. 

• If the students have already had a thermodynamics course (although I do not 
recommend this course sequence). Part II could be skipped entirely. However, 
even if this choice is made, students might still find Chapters 9 to 18 useful for 
review. 

One possibility that I do not recommend would be to skip the computational 
material. I am strongly of the opinion that the undergraduate physics curricula at 
most universities still contain too little instruction in the computational methods that 
students will need in their careers. 
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Introduction 


If, in some cataclysm, all scientific knowledge were to be destroyed, and only 
one sentence passed on to the next generation of creatures, what statement 
would contain the most information in the fewest words? I believe it is the 
atomic hypothesis (or atomic fact, or whatever you wish to call it) that 
all things are made of atoms—little particles that move around in perpetual 
motion, attracting each other when they are a little distance apart, but repelling 
upon being squeezed into one another. In that one sentence you will see an 
enormous amount of information about the world, if just a little imagination 
and thinking are applied. 

Richard Feynman, in The Feynman Lectures on Physics 


1.1 Thermal Physics 

This book is about the things you encounter in everyday life: the book you are reading, 
the chair on which you are sitting, the air you are breathing. It is about things that 
can be hot or cold; hard or soft; solid, liquid, or gas. It is about machines that work 
for you: automobiles, heaters, refrigerators, air conditioners. It is even about your 
own body and the stuff of life. The whole topic is sometimes referred to as thermal 
physics, and it is usually divided into two main topics: thermodynamics and statistical 
mechanics. 

Thermodynamics is the study of everything connected with heat. It provides 
powerful methods for connecting observable quantities by equations that are not 
at all obvious, but are nevertheless true for all materials. Statistical mechanics is 
the study of what happens when large numbers of particles interact. It provides 
a foundation for thermodynamics and the ultimate justification of why thermody¬ 
namics works. It goes beyond thermodynamics to reveal deeper connections between 
molecular behavior and material properties. It also provides a way to calculate the 
properties of specific objects, instead of just the universal relationships provided by 
thermodynamics. 

The ideas and methods of thermal physics differ from those of other branches of 
physics. Thermodynamics and statistical mechanics each require their own particular 
ways of thinking. Studying thermal physics is not about memorizing formulas; it is 
about gaining a new understanding of the world. 
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1.2 What are the Questions? 

Thermal physics seeks quantitative explanations for the properties of macroscopic 
objects, where the term ‘macroscopic’ means two things: 

1. A macroscopic object is made up of a large number of particles. 

2. Macroscopic measurements have limited resolution. 

The apparently vague terms ‘large number’ and ‘limited resolution’ take on specific 
meanings with respect to the law of large numbers that we will study in Chapter 3. We 
will see that the relative statistical uncertainty in quantities like the energy and the 
number of particles is usually inversely proportional to the square root of the number 
of particles. If this uncertainty is much smaller than the experimental resolution, it can 
be neglected. This leads to thermodynamics, which is a description of the macroscopic 
properties that ignores small statistical fluctuations. 

We will assume that we somehow know what the object is made of; that is, what 
kinds of atoms and molecules it contains, and how many of each. We will generally 
speak of ‘systems’ instead of objects. The difference is that a system can consist of any 
number of objects, and is somehow separated from the rest of the universe. We will 
concentrate on systems in equilibrium; that is, systems that have been undisturbed 
long enough for their properties to take on constant values. We shall be more specific 
about what ‘long enough’ means in Chapter 21. 

In the simplest case—which we will consider in Part I—a system might be com¬ 
pletely isolated by ‘adiabatic’ walls; that is, rigid barriers that let nothing through. 
We will also assume, at least at first, that we know how much energy is contained in 
our isolated system. 

Given this information, we will ask questions about the properties of the system. 
We will first study a simple model of a gas, so we will want to know what temperature 
and pressure are, and how they are related to each other and to the volume. We will 
want to know how the volume or the pressure will change if we heat the gas. As we 
investigate more complicated systems, we will see more complicated behavior and And 
more questions to ask about the properties of matter. 

As a hint of things to come, we will And that there is a function of the energy E, 
volume V, and number of particles N that is sufficient to answer all questions about 
the thermal properties of a system. It is called the entropy, and it is denoted by the 
letter S. 

If we can calculate the entropy as a function of E, V, and N, we know everything 
about the macroscopic properties of the system. For this reason, S = S{E,V,N) is 
known as a fundamental relation, and it is the focus of Part I. 

1.3 History 

Atoms exist, combine into molecules, and form every object we encounter in life. Today, 
this statement is taken for granted. However, in the nineteenth century, and even into 
the early twentieth century, these were fighting words. The Austrian physicist Ludwig 
Boltzmann (1844-1906) and a small group of other scientists championed the idea of 
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atoms and molecules as the basis of a fundamental theory of the thermal properties 
of matter, against the violent objections of another Austrian physicist, Ernst Mach 
(1838-1916), and many of his colleagues. Boltzmann was right, of course, but the issue 
had not been completely settled even at the time of his tragic suicide in 1906. In the 
tradition of Boltzmann, the intention in this book is to present thermodynamics as a 
consequence of the molecular nature of matter. 

The theory of thermodynamics was developed without benefit of the atomic 
hypothesis. It began with the seminal work of Sadi Carnot (French physicist, 
1792-1832), who initiated the scientific discussion of heat engines. The First Law 
of Thermodynamics was discovered by James Prescott Joule (English physicist, 
1818-1889), when he established that heat was a form of energy and measured the 
mechanical equivalent of heat. The Second Law of Thermodynamics was first stated 
in 1850 by Rudolf Clausius (German physicist, 1822-1888), who in 1865 also invented 
the related concept of entropy. The Second Law can be expressed by the statement 
that the entropy of an isolated system can increase, but not decrease. 

Entropy was a mystery to nineteenth-century scientists. Clausius had given entropy 
an experimental definition that allowed it to be calculated, but its meaning was 
puzzling. Like energy it could not be destroyed, but unlike energy it could be created. 
It was essential to the calculation of the efficiency of heat engines (machines that turn 
the energy in hot objects into mechanical work), but it did nt seem to be related to 
any other physical laws. 

The reason why scientists working in the middle of the nineteenth century found 
entropy so mysterious is that few of them thought in terms of atoms or molecules. 
Even with molecular theory, explaining the entropy required brilliant insight; without 
molecular theory, there was no hope. 

Serious progress in understanding the properties of matter and the origins of 
thermodynamic laws on the basis of the molecular nature of matter began in the 1860s 
with the work of Boltzmann and the American physicist J. Willard Gibbs (1839-1903). 

Gibbs worked from a formal starting point, postulating that observable quantities 
could be calculated from an average over many replicas of an object in different 
microscopic configurations, and then working out what the equations would have to 
be. His work is very beautiful (to a theoretical physicist), although somewhat formal. 
However, it left certain questions unresolved—most notably, what has come to be 
called ‘Gibbs’ paradox.’ This issue concerned a discontinuous change in the entropy 
when differences in the properties of particles in a mixture were imagined to disappear 
continuously. Gibbs himself did not regard this as a paradox; it was so named by 
the German physicist Otto Wiedeburg (1866-1901), who had read Gibbs’ work on 
thermodynamics in a German translation by the prominent German chemist Wilhelm 
Ostwald (1853-1932, Nobel Prize 1909). The issues involved are still a matter of debate 
in the twenty-first century. 

Boltzmann devoted most of his career to establishing the molecular theory of 
matter and deriving the consequences of the existence of atoms and molecules. One 
of his great achievements was his 1877 definition of entropy—a definition which 
provides a physical interpretation of the entropy and the foundation of statistical 
mechanics. 
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Part I of this book is devoted to developing an intuitive understanding of the 
concept of entropy, based on Boltzmann’s definition. We will present an explicit, 
detailed derivation of the entropy for a simple model to provide insight into its signif¬ 
icance. Later parts of the book will develop more sophisticated tools for investigating 
thermodynamics and statistical mechanics, but they are all based on Boltzmann’s 
definition of the entropy. 

1.4 Basic Concepts and Assumptions 

This book is concerned with the macroscopic behavior of systems containing many 
particles. This is a broad topic, since everything we encounter in the world around 
us contains enormous numbers of atoms and molecules. Even in fairly small objects, 
there are 10^® or more atoms. 

Atoms and molecules are not the only examples of large numbers of particles in 
a system. Colloids, for example, can consist of 10^^ or more microscopic particles 
suspended in a liquid. The large number of particles in a typical colloid means that 
they are also well described by statistical mechanics. 

Another aspect of macroscopic experiments is that they have limited resolution. 
We will see in Part I that the fluctuations in quantities like the density of a system are 
approximately inversely proportional to the square root of the number of particles in 
the system. If there are 10^° molecules, this gives a relative uncertainty of about 10“^° 
for the average density. Because it is rare to measure the density to better than one 
part in 10®, these tiny fluctuations are not seen in macroscopic measurements. Indeed, 
an important reason for Mach’s objection to Boltzmann’s molecular hypothesis was 
that no direct measurement of atoms had ever been carried out in the nineteenth 
century. 

Besides the limited accuracy, it is rare for more than a million quantities to 
be measured in an experiment, and usually only a handful of measurements are 
made. Since it would take about 67V quantities to specify the microscopic state of 
TV atoms, thermodynamic measurements provide relatively little information about 
the microscopic state. 

Due to our lack of detailed knowledge of the microscopic state of an object, we need 
to use probability theory—discussed in Chapters 3 and 5—to make further progress. 
However, we do not know the probabilities of the various microscopic states either. 
This means that we have to make assumptions about the probabilities. We will make 
the simplest assumptions that are consistent with the physical constraints (number of 
particles, total energy, and so on): we will assume that everything we do not know is 
equally probable. 

Based on our assumptions about the probability distribution, we will calculate the 
macroscopic properties of the system and compare our predictions with experimental 
data. We will find that our predictions are correct. This is comforting. However, we 
must realize that it does not necessarily mean that our assumptions were correct. In 
fact, we will see in Chapter 21 that many different assumptions would also lead to the 
same predictions. This is not a flaw in the theory but simply a fact of life. Actually, 
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recognizing this fact helps a great deal in resolving apparent paradoxes, as we will see 
in Chapter 21. 

The predictions we make based on assumptions about the probabilities of micro¬ 
scopic states lead to the postulates of thermodynamics. These postulates, in turn, are 
sufficient to derive the very powerful formalism of thermodynamics, as we will see in 
Part II. 

These same assumptions about the probabilities of microscopic states also lead to 
the even more powerful formalism of statistical mechanics, which we will investigate 
in the last two parts of the book. 

1.4.1 State Functions 

It has long been known that when most macroscopic systems are left by themselves 
for a long period of time, their measurable properties stop changing and become time- 
independent. Simple systems, like a container of gas, evolve into macroscopic states 
that are well described by a small number of variables. For a simple gas, the energy, 
volume, and number of particles might be sufficient. These quantities, along with other 
quantities that we will discuss, are known as ‘state functions’. 

The molecules in a macroscopic system are in constant motion, so that the 
microscopic state is constantly changing. This fact leads to a basic question. How is it 
that the macroscopic state can be time-independent with precisely defined properties? 
The answer to this question should become clear in Part I of this book. 

1.4.2 Irreversibility 

A second basic question is this. Even if there exist equilibrium macroscopic states 
that are independent of time, how can a system evolve toward such a state but 
not away from it? How can a system obeying time-reversal-invariant laws of motion 
show irreversible behavior? This question has been the subject of much debate, at 
least since Johann Josef Loschmidt’s (Austrian physicist, 1821-1895) formulation of 
the ‘reversibility paradox’ in 1876. We will present a resolution of the paradox in 
Chapter 21. 

1.4.3 Entropy 

The Second Law of Thermodynamics states that there exists a state function called 
the ‘entropy’ that is maximized in equilibrium. Boltzmann’s 1877 definition of the 
entropy provides an account of what this means. A major purpose of the calculation 
of the entropy of a classical ideal gas in Part I is to obtain an intuitive understanding 
of the significance of Boltzmann’s entropy. 

1.5 Road Map 

The intention of this book is to present thermal physics as a consequence of the 
molecular nature of matter. The book is divided into four parts, in order to provide 
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a systematic development of the ideas for someone coming to the subject for the first 
time, as well as for someone who knows the basic material but would like to review a 
particular topic. 

Part I : Entropy The ideal gas is a simple classical model with macroscopic prop¬ 
erties that can be calculated exactly. This allows us to derive the entropy in closed 
form without any hidden assumptions. Since the entropy of the classical ideal gas 
exhibits most of the thermodynamic properties of more general systems, it will 
serve as an introduction to the otherwise rather abstract and formal postulates of 
thermodynamics. In the last two chapters of Part I, the formal expression for the 
entropy of a classical gas with interacting particles is obtained, along with general 
expressions for the temperature, pressure, and chemical potential, which establish the 
foundations of classical statistical mechanics. 

Part II: Thermodynamics The formal postulates of thermodynamics are intro¬ 
duced, based on the properties of the entropy derived in Part I. Our treatment follows 
the vision of Laszlo Tisza (Hungarian physicist, 1907-2009) and Herbert B. Callen 
(American physicist, 1919-1993). Although the original development of thermodynam¬ 
ics by nineteenth-century physicists was a brilliant achievement, their arguments were 
somewhat convoluted because they did not understand the microscopic molecular basis 
of the laws they had discovered. Deriving the equations of thermodynamics from postu¬ 
lates is much easier than following the historical path. The full power of thermodynam¬ 
ics can be developed in a straightforward manner and the structure of the theory made 
transparent. 

Part III: Classical statistical mechanics Here we return to classical statistical 
mechanics to discuss more powerful methods of calculation. In particular, we introduce 
the canonical ensemble, which describes the behavior of a system in contact with a 
heat bath at constant temperature. The canonical ensemble provides a very powerful 
approach to solving most problems in classical statistical mechanics. We also introduce 
the grand canonical ensemble, which describes a system that can exchange particles 
with a large system at a fixed chemical potential. This ensemble will prove to be par¬ 
ticularly important when we encounter it again in Chapters 26 through 29 of Part IV, 
where we discuss quantum systems of indistinguishable particles. Statistical mechanics 
can derive results that go beyond those of thermodynamics. We discuss and resolve 
the apparent conflict between time-reversal-invariant microscopic equations with the 
obvious existence of irreversible behavior in the macroscopic world. Part HI will also 
introduce molecular dynamics and Monte Carlo computer simulations to demon¬ 
strate some of the modern methods for obtaining information about many-particle 
systems. 

Part IV: Quantum statistical mechanics In the last part of the book we develop 
quantum statistical mechanics, which is necessary for the understanding of the prop¬ 
erties of most real systems. After two introductory chapters on basic ideas we will 
devote chapters to black-body radiation and lattice vibrations. There is a chapter 
on the general theory of indistinguishable particles, followed by individual chapters 
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on the properties of bosons and fermions. Since the application of the theory of 
fermions to the properties of insulators and semiconductors has both theoretical and 
practical importance, this topic has a chapter of its own. The last chapter provides 
an introduction to the Ising model of ferromagnetism as an example of the theory of 
phase transitions. 
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The Classical Ideal Gas 


A mathematician may say anything he pleases, but a physicist must be at least 
partially sane. 

J. Willard Gibbs 


The purpose of Part I of this book is to provide an intuitive understanding of the 
entropy, based on calculations for a simple model. The model chosen is the classical 
ideal gas, for which the entropy can be calculated explicitly and completely without 
any approximations or hidden assumptions. 

The treatment is entirely in terms of the theory of classical mechanics. No quantum 
mechanical concepts are used. All the ideas will follow directly from Boltzmann’s work. 
We will use more modern mathematical methods than he did to derive the entropy 
of the classical ideal gas, but we will make no assumptions with which he was not 
familiar. 

In Chapters 7 and 8 the formal expression for the entropy will be extended to 
classical systems with interacting particles. Although the expression we obtain can 
rarely be evaluated exactly, the formal structure will be sufficient to provide a basis 
for the development of thermodynamics in Part II. The same formal structure will 
also lead to more powerful methods of calculation for statistical mechanics in Parts III 
and IV. 


2.1 Ideal Gas 

What distinguishes an ‘ideal’ gas from a ‘real’ gas is the absence of interactions 
between the particles. Although an ideal gas might seem to be an unrealistic model, its 
properties are experimentally accessible by studying real gases at low densities. Since 
even the molecules in the air you are breathing are separated by an average distance 
of about ten times their diameter, nearly ideal gases are easy to find. 

The most important feature that is missing from a classical ideal gas is that it does 
not exhibit any phase transitions. Other than that, its properties are qualitatively the 
same as those of real gases, which makes it valuable for developing intuition about 
statistical mechanics and thermodynamics. 

The great advantage of the ideal gas model is that all of its properties can be 
calculated exactly, and nothing is obscured by mathematical complexity. 
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2.2 Phase Space of a Classical Gas 

Our model of a classical gas consists of N particles contained in some specified 
volume. Each particle has a well-defined position and momentum. The positions of 
every particle can be represented as a point in configuration space—an abstract 3iV- 
dimensional space, with axes for every coordinate of every particle. These coordinates 
can be given in various forms. 

q={ri\t = l,...,N} (2.1) 

= {xi,yi,Zi\i = 1,..., A^} 

= {qj\j = l,...,3N} 

The momentum of every particle can be represented as a point in momentum 
space—an abstract SA'^-dimensional space, with axes for every component of the 
momentum of every particle. 


p={p,\t = l,...,N} (2.2) 

= {Px,i}Py,iiPy,i\'i' — Ij • ■ • j 

= {Pj\j = • ■ ,3iV} 

The complete microscopic state of the system can be described by a point in phase 
space—an abstract GA'^-dimensional space with axes for every coordinate and every 
momentum component for all N particles. Phase space is the union of configuration 
space and momentum space, {p,q}. 


{P,q} = {q 3 ,Pj\j = 1,- ■ •,3iV} 


(2.3) 


The kinetic energy of the f-th particle is given by the usual expression, |pip/2m, 
and the total kinetic energy is just the sum of the kinetic energies of all particles. 


N 

^ = E 


2 = 1 


2m 


(2.4) 


Since, by definition, there are no interactions between the particles in an ideal gas, 
the potential energy of the system is zero. 


2.3 Distinguishability 

Particles will be regarded as distinguishable, in keeping with classical concepts. To 
be specific, particles are distinguishable when the exchange of two particles results in 
a different microscopic state. In classical mechanics, this is equivalent to saying that 
every point in phase space represents a different microscopic state. Distinguishability 
does not necessarily mean that the particles have different properties; classically, 
particles were always regarded as distinguishable because their trajectories could, at 
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least in a thought experiment, be followed and the identity of individual particles 
determined. 

On the other hand, it will be important to remember that experiments on macro¬ 
scopic systems are always assumed to have limited resolution. In both statistical 
mechanics and thermodynamics we are concerned with measurements that do not 
have the sensitivity to resolve the positions or identities of individual atoms. 


2.4 Probability Theory 

Because of the importance of probability theory in statistical mechanics, two chapters 
are devoted to the topic. The chapters discuss the basic principles of the probability 
theory of discrete random variables (Chapter 3) and continuous random variables 
(Chapter 5). 

The mathematical treatment of probability theory has been separated from the 
physical application for several reasons: (1) it provides an easy reference for the 
mathematics, (2) it makes the derivation of the entropy more compact, and (3) it 
is unobtrusive for those readers who are already completely familiar with probability 
theory. 

If the reader is already familiar with probability theory. Chapters 3 and 5 might be 
skipped. However, since the methods for the transformation of random variables pre¬ 
sented there differ from those usually found in mathematics textbooks, these chapters 
might still be of some interest. It should be noted that we will need the transformation 
methods using Dirac delta functions, which are rarely found in mathematical texts on 
probability theory. 

The chapters on probability theory are placed just before the chapters in which the 
material is first needed to calculate contributions to the entropy. Chapter 3 provides 
the methods needed to calculate the contributions of the positions in Chapter 4, and 
Chapter 5 provides the methods needed to calculate the contributions of the momenta 
in Chapter 6. 

To apply probability theory to the calculation of the properties of the classical ideal 
gas—or any other model, for that matter—we will have to make assumptions about 
the probability distribution of the positions and momenta of 10^° or more particles. 
Our basic strategy will be to make the simplest assumptions consistent with what we 
know about the system and then calculate the consequences. 

Another way of describing our strategy is that we are making a virtue of our 
ignorance of the microscopic states and assume that everything we don’t know is 
equally likely. How this plays out in practice is the subject of the rest of the book. 


2.5 Boltzmann's Definition of the Entropy 

In 1877, after a few less successful attempts, Boltzmann defined the entropy in terms of 
the probability of a macroscopic state. His explanation of the Second Law of Thermo¬ 
dynamics was that isolated systems naturally develop from less probable macroscopic 
states to more probable macroscopic states. Although Boltzmann’s earlier efforts to 
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prove this with his famous H-theorem were problematic and highly controversial, his 
basic insight is essentially correct. 

In his 1877 paper, Boltzmann also specified that the entropy should be defined in 
terms of a composite system; that is, a system composed of two or more subsystems 
with some sort of constraint. An example of such a composite system would be a 
volume of gas divided into two smaller volumes (or subsystems) by a partition. The 
partition acts as a constraint by restricting the number of particles in each subsystem 
to be constant. The removal of the partition would then allow the system to develop 
from a less probable macroscopic state to a more probable macroscopic state. The final 
state, after the composite system had come to equilibrium, would correspond to the 
most probable macroscopic state. According to the Second Law of Thermodynamics, 
the thermodynamic entropy should also be maximized in the equilibrium state. The 
comparison of these two properties of the equilibrium state led Boltzmann to associate 
the entropy with the probability of a macroscopic state, or more precisely with the 
logarithm of the probability. 

In the following chapters we will make a direct application of Boltzmann’s definition 
to the calculation of the entropy of the classical ideal gas, to within additive and 
multiplicative constants that we will determine later. 

2.6 S = k\ogW 

Boltzmann’s achievement has been honored with the inscription of the equation 
S = /clogIT on his tombstone. The symbol S denotes the entropy. The symbol W 
denotes the German word Wahrscheinlichkeit, which means ‘probability’. Curiously 
enough, Boltzmann never wrote this equation, although it does accurately reflect his 
ideas. The equation was first written in this form by the German physicist Max Planck 
(1858-1947) in 1900. The constant k, also written as ks, is known as the Boltzmann 
constant. 

The symbol W has often been misinterpreted to mean a volume in phase space, 
which has caused a considerable amount of trouble. This misinterpretation is so 
common that many scientists are under the impression that Boltzmann defined the 
entropy as the logarithm of a volume in phase space. Going back to the original 
meaning of W and Boltzmann’s 1877 definition eliminates much of the confusion 
about the statistical interpretation of entropy. 

The main differences between Boltzmann’s treatment of entropy and the one in 
this book lie in the use of modern mathematical methods and the explicit treatment 
of the dependence of the entropy on the number of particles. 

2.7 Independence of Positions and Momenta 

In the derivation of the properties of the classical ideal gas, we will assume that 
the positions and momenta of the particles are independent. We will present a more 
formal definition of independence in the next chapter, but the idea is that knowing the 
position of a particle tells us nothing about its momentum, and knowing its momentum 
tells us nothing about its position. As demonstrated at the beginning of Ghapter 4, 
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the independence of the positions and momenta means that their contributions to 
the total entropy can be calculated separately and simply added to produce the final 
answer. 

2.8 Road Map for Part I 

The analysis of the entropy in Part I has been divided into chapters to make it easier 
to keep track of the different aspects of the derivation. 

The concepts and equations of discrete probability theory are developed in 
Chapter 3, just before it is needed in Chapter 4 to calculate the contributions of 
the positions to the entropy. 

Probability theory for continuous random variables is discussed in Chapter 5, just 
before its application to the calculation of the contributions of the momenta to the 
entropy in Chapter 6. 

Chapter 7 generalizes the results to systems with interacting particles. 

Chapter 8 completes the foundations of classical statistical mechanics by relating 
the partial derivatives of the entropy to the temperature, pressure, and chemical 
potential. 

The following flow chart is intended to illustrate the organization of Part I. The 
derivation of the entropy of the classical ideal gas follows the arrows down the right- 
hand side of the flow chart. 


Flow chart for Part I 
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Discrete Probability Theory 


It may he that the race is not always to the swift nor the battle to the 
strong—but that’s the way to bet. 

Damon Runyon 


3.1 What is Probability? 

The definition of probability is sufficiently problematic to have occasioned something 
akin to religious wars in academic departments of statistics. There are two basic 
factions: frequentists and Bayesians. 

To understand why the definition of probability can be a problem, consider an 
experiment with N trials, each of which can result in a success or failure by some 
criterion. If Ng be the number of successes, then Ng/N is called the frequency of 
success for those trials. 

Frequentists define the probability by the asymptotic frequency of success in the 
limit of an infinite number of trials. 


P = 


lim 

N —>-oo 


N 


(3.1) 


This definition looks precise and objective. Indeed, it is found as the defin¬ 
ition in many physics texts. The major problem is that humans have a finite 
amount of time available, so that by the frequentist definition we can never deter¬ 
mine the probability of anything. Bayesian probability provides a solution to this 
quandary. 

The Bayesian view of probability is based on the work of Thomas Bayes (English 
mathematician and Presbyterian minister, 1702-1761). Bayesians define probability 
as a description of a person’s knowledge of the outcome of a trial, based on whatever 
evidence is at that person’s disposal. 

One great advantage of the Bayesian definition of probability is that it gives a clear 
meaning to a statement such as: ‘The mass of a proton is 1.672621637(83) x kg’, 
where the ‘(83)’ is called the uncertainty in the last digit. What does ‘(83)’ mean? 
Certainly the mass of a proton is an unambiguous number that does not take on 
different values for different trials. Nevertheless, the ‘(83)’ does have meaning as an 
expression of our uncertainty as to the exact value of a proton’s mass. 
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Bayesian probability is accepted by most statisticians. However, it is in disrepute 
among some physicists because they regard it as subjective, in the sense that it 
describes what an individual knows, rather than being absolute truth. However, none 
of us has access to absolute truth, and Bayesian statistics provides an appropriate way 
to describe what we learn from experiments. 

In my opinion, the only reasonable form of objectivity that we can demand is 
that two observers with the same information will come to the same conclusions, and 
Bayesian statistics fulfills this requirement. 

There is one other use of the word ‘probability’ that can be quite important. I shall 
call it ‘model probability’, and it differs from both the frequentist and the Bayesian 
meanings. 

A model probability is an assumption (or guess), as to what the frequency of 
success would be for an infinite number of trials. The assumption is usually based on 
the plausible argument that events that look very much alike, have equal probabilities. 
We can then use this assumption, or model, to calculate the predicted outcome of an 
experiment. If measurements agree with our predictions, we can say that our model 
is consistent with the experiment. This is not the same thing as saying that our 
model probabilities are correct, and we will see later that many different assumptions 
can lead to the same predictions. Nevertheless, agreement with experiment is always 
comforting. 

Statistical mechanics is based on very simple assumptions, expressed as model 
probabilities, that lead to a wide variety of predictions in excellent agreement with 
experiment. How this is done is the main subject of this book. 

All three definitions of probability have the same mathematical structure. This 
reduces the amount of mathematics that we have to learn, but unfortunately does not 
resolve the controversies. 


3.2 Discrete Random Variables and Probabilities 

We begin by defining a set of elementary events 

A = {a,\j = l,NA} (3.2) 

and assigning a probability P{a,j) to each event. The combination of random events 
and their probabilities is called a ‘random variable’. If the number of elementary events 
is a finite or countable, it is called a ‘discrete random variable’. 

The probabilities must satisfy the conditions that 


0 < P{aj) < 1 (3.3) 

for all Qj. An impossible event has probability zero, and a certain event has 
probability 1. 

Random events can be anything: heads/tails, red/black, and so on. If the random 
events are all numbers, the random variable is called a ‘random number’. 


18 Discrete Probability Theory 


Elementary events are defined to be exclusive—one, and only one, event can occur 
at each trial—so that the probabilities must also satisfy the normalization condition 

Na 

Y^P{a,) = l (3.4) 

i=i 

To simplify notation we will often write this equation as 

^P(a) = l (3.5) 

a 

suppressing explicit mention of the number of elementary events. 

3.3 Probability Theory for Multiple Random Variables 

If more than one thing can happen at each trial we can describe the situation with 
two or more sets of random events. For example, both an event from A and an event 
from 


B = {bk\k = l,NB} (3.6) 

might occur. We can then define a joint probability P{aj^bk )—or more simply, 
P{a, b )—which must satisfy 


0 < P{a,b) < 1 

(3.7) 

^^P(a,6) = 1 

(3.8) 


a b 


3.3.1 Marginal and Conditional Probability 

Naturally, if we have P(a, b) we can retrieve the information for either A or B alone. 
The marginal probability of A is defined by 

PA(a) = ^P(a,6) (3.9) 

b 

with a similar expression for Pb (&). A nice feature of marginal probabilities is that they 
automatically satisfy the positivity and normalization criteria in eqs. (3.3) and (3.4). 

The name marginal probability comes from the practice of calculating it in the 
margins of a table of probabilities. Table 3.1 shows an example for two random 
variables that each take on two values. 

Conditional probability is the probability of an event a, given that event b has 
occurred. It is denoted by P{a\b), and is related to the full probability of both A and 
B occurring by the equations: 
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P{a,b) = P{a\b)PB{b) 
= P{b\a)PA{a) 


If Psib) 7 ^ 0, the conditional probability P{a\b) can be written as 


P{a\b) 


P{a, b) 

PB{b) 


Combining eqs. (3.10) and eqs. (3.11) gives us Bayes’ theorem. 


P{a\b) 


P{b\a)PA{a) 

Psib) 


We will discuss some of the consequences of Bayes’ theorem in Section 5.5. 


(3.10) 

(3.11) 


(3.12) 


(3.13) 


3.3.2 Independent Variables 

A particularly interesting case occurs when the probability distribution can be written 
as a product: 


P{a,b) = PA{a)PB{b) 


(3.14) 


When eq. (3.14) is true, the two random variables are said to be independent because 
the conditional probability P(a\b) is then independent of b, 


P{a\b) 


P{a, b) 
PB{b) 


PA{a)PB{b) 

PB{b) 


PA{a) 


(3.15) 


and P{b\a) is independent of a. 

Table 3.1 gives an example of independent random variables, and Table 3.2 gives 
an example of random variables that are not independent. 


3.3.3 Pairwise Independence and Mutual Independence 

If we have more than two random variables, the set of random variables might satisfy 
two kinds of independence: pairwise or mutual. 


Table 3.1 Example of a table of probabilities for independent random variables: The events of A 
(labelled ‘3’ and ‘4’) are listed down the left column, and events of B (labelled ‘P and ‘2’) across 
the top. The values of the probabilities are given in the four center squares. The margins contain 
the marginal probabilities, as defined in eq. (3.9). 


A\B 

1 

2 

PA{a) 

3 

1/2 

1/4 

3/4 

4 

1/6 

1/12 

1/4 

PB{b) 

2/3 

1/3 

1 
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Table 3.2 An example of a table of probabilities for random variables that are not independent. 
The table is arranged in the same way as Table 3.1. 


a\b 

1 

2 

PA{a) 

3 

0.5 

0.25 

0.75 

4 

0.1 

0.15 

0.25 

Psib) 

0.6 

0.4 

1 


Pairwise independence means that the marginal distribution of any pair of random 
variables can be written as the product of the marginal distributions of the individual 
random variables. 

Mutual independence means that the marginal distribution of any subset of random 
variables can be written as the product of the marginal distributions of the individual 
random variables. 

It is obvious that mutual independence implies pairwise independence. Whether 
the converse is true is the subject of a problem in this chapter. 


3.4 Random Numbers and Functions of Random Variables 

Given an arbitrary random variable A = {aj\j = 1, ■ ■ ■ Na}, we can define a numer¬ 
ical function on the set of elementary events, F = {P"(aj)|l, • • • ,Na}. The set 
of random numbers F, together with their probabilities, is then also a random 
number. 

If we introduce the Kronecker delta 


(5 


r:,y ~ 


1 x = y 

0 x^y 


(3.16) 


then we can write the probability distribution of F compactly. 


Ppif) =y^Jf^Fia)PAia) 

a 


(3.17) 


As a simple illustration, consider a random variable that takes on the three values 
— 1, 0, and 1, with probabilities P^(—1) = 0.2, Fyi(O) = 0.3, and Pa{P) = 0.5. Define 
a function F[a) = |a|, so that F takes on the values 0 and 1. The probabilities Ppif) 
are found from eq. (3.17). 


1 

Pf{0) = ^ ^0,F{a)PA{a) 

a—— 1 

= ^o,f{-i)Pa{—^) + ^o,f{o)Pa{0) + ^o,f{i)Pa{^) 
= 0-k P(0)-k 0 = 0.3 


(3.18) 
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Pf{ 1) = Y, Si,Fia)PAia) (3.19) 

a—— 1 

= —1) + <5i,F(O)^a(0) + (^i^f(I)-Pa(I) 

= Pa{-1) + 0 + Pa{1) = 0.2 + 0.5 = 0.7 

We can also use the Kronecker delta to express arbitrary combinations of random 
numbers to form new compound random numbers. For example, if X and Y are random 
numbers, and G(x, y) is an arbitrary function, we can define a new random variable, 
G. The probability distribution of G is given by a sum over all combinations of the 
events of X and Y, with the Kronecker delta picking out the ones that correspond to 
particular events of G. 


Pcig) = YY1 ^g,GG,y)Pi.X: y) ( 3 - 20 ) 

X y 

A warning is necessary because the limits on the sums in eq. (3.20) have been 
suppressed. The only difficult thing about actually doing the sums is in keeping track 
of those limits. Since being able to keep track of limits will also be important when 
we get to continuous distributions, we will illustrate the technique with the simple 
example of rolling two dice and asking for the probability distribution of their sum. 


Example: Probability of the Sum of Two Dice 

We will assume that the dice are honest, which they tend to be in physics problems, if 
not in the real world. Let X = {x\x = 1,2,3,4,5, 6} be the random number represent¬ 
ing the outcome of the first die, with Y a corresponding random number for rolling 
the second die. The sum S = X + Y. The values taken on by S range from 2 to 12. 
Since all elementary events are equally likely. 


p(».,) = PxWPHx) = (l) (1) = F 


(3.21) 


eq. (3.20) then becomes 

6 6 

= (3.22) 

x—1y—1 

Do the sum over y first. Its value depends on whether s = x -\- y for some value of 
y, or equivalently, whether s — a: is in the interval [1,6]. 




,x+y 


y=i 


ll<s — a;<6 
0 otherwise 


(3.23) 


This places two conditions on the remaining sum over x. Only those values of x for 
which both x < s — 1 and x > s — 6 contribute to the final answer. These limits are 
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Table 3.3 Determining the limits for the second sum when evaluating eq. (3.22). 



lower limit 

upper limit 

Limits on sum: 

X > 1 

X < 6 

From the Kronecker delta: 

X > s — 6 

X < s — 1 

More restrictive if s < 7 

X > 1 

X < s — 1 

More restrictive if s > 7 

X > s — 6 

X < 6 





in addition to the limits of x < 6 and x > 1 that are already explicit in the sum on X. 
Since all four of these inequalities must be satisfied, we must take the more restrictive 
of the inequalities in each case. Which inequality is the more restrictive depends on 
the value of s, as indicated in Table 3.3. 

For s < 7, the lower bound on x is 1 and the upper bound is s — 1. For s > 7, the 
lower bound on x is s — 6 and the upper bound is 6. The sums can then be evaluated 
explicitly. 


(s-l 

E l s — 1 
36 ^ 36 


s < 7 


P{s) = 


1 13 - s 

— = — s > 7 


a;=s—6 


36 


(3.24) 


Clearly, even though the probability distribution of both X and Y are uniform, the 
probability distribution of S is not. This is an important result. Although we will be 
generally assuming a uniform underlying probability distribution for most microscopic 
quantities, we will see that the distribution of macroscopic observables will be very 
sharply peaked. 

It must be admitted that there are easier ways of numerically computing the 
probability distribution of the sum of two dice, especially if a numerical result is 
required, eq. (3.20) is particularly easy to evaluate in a computer program, since the 
Kronecker delta just corresponds to an ‘IF’-statement. However, we will have ample 
opportunity to apply the method just described to problems for which it is the simplest 
approach. One reason why the example in this section was chosen is that the correct 
answer is easy to recognize, which makes the method more transparent. 


3.5 Mean, Variance, and Standard Deviation 

The mean, or average, of a function F(A), defined on the random variable A, is given by 

{F)='£F{a)PA{a) (3.25) 
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Similarly, the n-th moment of F can be defined as 


{F-) = Y^F{arPA{a) 

a 


(3.26) 


The n-th central moment is defined by subtracting the mean before taking the 
n-th power in the definition. 


((F - (F))") = ^ (F(a) - (F))" PA{a) (3.27) 

a 


The second central moment plays an important role in statistical analysis and is 
called the variance, cr^. 

= ((F - (F))^) = ^ (F(a) - {F)f PA{a) (3.28) 

a 


It can also be written as 


= {{F - {F)f) = (F^) - {Ff (3.29) 


The square root of the variance is called the standard deviation. 

a = \/(F2) - (F)2 (3.30) 

The standard deviation is frequently used as a measure of the width of a probability 
distribution. 


3.6 Correlation Functions 

Suppose we have two random numbers, A and B, and their joint probability distribu¬ 
tion, P{A, B), along with functions F{A) and G{B) defined on the random variables. 
F and G are random numbers, and we can ask questions about how they are related. 
In particular, we can define a correlation function, fpG- 

fFG = {FG)-{F){G) (3.31) 

If F and G are independent random numbers, we would expect the correlation 
function to vanish, which it does. 
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fpG = EE F{a)G{b)P{a,b) (3.32) 

a b 

-Y,F{a)PA{a)J2G{b)PB{b) 

a b 

= EE F{a)G{b)PA{a)PB{b) 

a b 

-^F(a)FA(a)^G(fo)PB(6) 

a b 

= 0 


3.7 Sets of Independent Random Numbers 

Given a set of random numbers {Fj\j = 1, - • ■ ,N}, we are often interested in a new 
random number formed by taking the sum. 

N 

S = J2Fj (3-33) 

i=i 

We can easily calculate the mean of the random number S, which is just the sum 
of the means of the individual random numbers. 

/ Af \ N 

(^) = (E^W=E(^:^) (3.34) 

\i=i / i=i 

If the random numbers are pairwise independent, we can also calculate the variance 
and the standard deviation. 

^ (5^) - (5)2 (3.35) 

N N N N 

= EE(^:^-^'^)-E(^:^-)E(^'=) 

j—1 k—1 j—1 k—1 

N N N N N 

= E E {f,){f,) + Y^{f^)-Y^{f,)Y^{f,) 

N 

= E((^/)-(^^)') 

N 

= E-I 

f=i 
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In this derivation, cr| denotes the variance of the j-th random number. We see that 
the variance of the sum of a set of pairwise independent random numbers is just the 
sum of the variances. 

If the random numbers Fj all have the same mean and variance, these equations 
simplify further. If {Fj) = {F) for all j, then 

N 

{S) = = N{F) (3.36) 

i=i 

Similarly, if cr| = for all j, then 

N 

= = (3.37) 

f=i 

Note that the standard deviation of S grows as the square root of the number of 
variables. 


CTs = cr 


Vn 


(3.38) 


On the other hand, the relative standard deviation decreases with the square root of 
the number of variables. 


as a^/N a 

M ~ {f)Vn 


(3.39) 


It might be argued that this is the most important result of probability theory for 
statistical mechanics. For many applications in statistical mechanics, the appropriate 
value of N is 10^° or higher, so that the relative uncertainties for macroscopic 
quantities are generally of the order of 10“^° or smaller. This is far smaller than 
most experimental errors, leading to predictions with no measurable uncertainty. 


3.8 Binomial Distribution 

A particularly important case is that of N independent, identically distributed random 
numbers, {Fj}, that can each take on the value 1 with probability p and 0 with 
probability 1 — p. An example would be N flips of a coin, where the coin might be fair 
{p = 0.5) or biased {p ^ 0.5). 

The mean and variance of each random number are easily seen to be {F) = p and 
cr^ = p{l — p). The mean and variance of the sum 

N 

S = Y,F, (3.40) 


are then 


{S) = pN 


( 3 . 41 ) 
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and 


= -P(l -p)N 


(3.42) 


The standard deviation is 


as = ^/p{l-p)N 


(3.43) 


and the relative standard deviation is 


o-s ^ v^p(l -p)jV 
(S) ~ pN 



(3.44) 


3.8.1 Derivation of the Binomial Distribution 

We can go further and calculate the explicit probability distribution P{S) of the sum 
of random numbers. This result will be extremely important in the analysis of the 
classical ideal gas. 

The probability of a specific subset of n random variables taking on the value 1, 
while the remaining N — n random variables that take on the value 0 is easily seen 
to be 


pn^l-p)N-n 


(3.45) 


To complete the calculation, we only need to determine the number of permutations 
of the random variables with the given numbers of Is and zeros. This number is the 
same as the number of ways by which N distinct objects can be put into two boxes, 
such that n of them are in the first box and N — n are in the second. 

To calculate the number of permutations, first consider the simpler problem of 
calculating the number of ways in which N distinct objects can be ordered. Since any 
of the N objects can be first, iV — 1 can be second, and so on, there are a total of 
N\ = N{N — 1){N — 2) • • • , 2,1 permutations. 

For our problem of putting objects into two boxes, the order of the objects in 
each box does not matter. Therefore, we must divide by n! for over counting in the 
first box and by (IV — n)! for over counting in the second box. The final number of 
permutations is known as the binomial coefficient and has its own standard symbol. 



(3.46) 


Multiplying by the probability given in eq. (3.45) gives us the binomial distribution. 


P(n|jV)= f- p»(l-p)^- 
nl{N — n)l 


((()p"(i-p) 


JV-n 


(3.47) 
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The binomial distribution acquires its name from the binomial theorem, which 
states that for any numbers p and q, 


N 


{p+qr = j2 


Nl 


N 


n\(N — n)\ 

n=0 ^ ' 


p^qN-n 


= E 

n—O 


N' 


p^qN-n 


(3.48) 


Setting q= 1— p proves that the binomial distribution in eq. (3.47) is normalized. 


3.8.2 Useful Identities for the Binomial CoefRcients 

Although the evaluation of the binomial coefficients appears to be straightforward, 
A^! grows so rapidly as a function of N that the numbers are too large for a direct 
application of the definition. For example, a popular spreadsheet overflows at A = 171, 
and my hand calculator cannot even handle A = 70. 

On the other hand, the binomial coefficients themselves do not grow very rapidly 
with A. The following identities, which follow directly from eq. (3.46), allow us to 
calculate the binomial coefficients for moderately large values of n and A without 
numerical difficulties. 



\n + ly n + 1 \n) 


The proofs of these identities will be left as exercises. 


(3.49) 

(3.50) 

(3.51) 


3.9 Gaussian Approximation to the Binomial Distribution 

For a fixed value of p and large values of A, the binomial distribution can be 
approximated by a Gaussian function. This is known as the central limit theorem. 
We will not prove it, but we will show how to determine the appropriate parameters 
in the Gaussian approximation. 

Gonsider a general Gaussian function. 


9{x) 


1 


■\/27rtT^ 


exp 


{x - XoY 
2a2 


(3.52) 


The mean and the location of the maximum coincide. 
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The variance is given by the second central moment. 


({x - Xof) = [ {x-Xofexp 

\ ' v2'Ka‘^ 7-00 


{x - Xo) 


21 


2a2 


dx 


\/2jr(T^ J-c 


exp 


2 1 


y 

'2a2 


dy 


V2'!ra‘^ 


V^[2cr 


2i3/2 




/ttV 


(3.54) 


It is now easy to find a Gaussian approximation to the binomial distribution, since 
both the mean and the variance are known exactly from eqs. (3.41) and (3.42). 


(n) = -pN 


(3.55) 


and 


0-2 = p(l — p)N 


(3.56) 


A Gaussian function with this mean and variance gives a good approximation to 
the binomial distribution for sufficiently large n and ISl. 


P{n\N) 


1 


Y^27rp(l — p)N . 2p{l—p)N_ 


{n — pNy 


(3.57) 


How large n and N must be for eq. (3.57) to be a good approximation will be left as 
a numerical exercise. 


3.10 A Digression on Gaussian Integrals 

To derive a good approximation for N\ that is practical and accurate for values of N 
as large as 10^°, we need to develop another mathematical tool: Gaussian integrals. 

Gaussian integrals are so important in statistical mechanics that they are well 
worth a slight detour. Although the formulas derived in this section can be found 
wherever fine integrals are sold, a student of statistical mechanics should be able to 
evaluate Gaussian integrals without relying on outside assistance. 

The first step in evaluating the integral 

/ OO 

(3.58) 

-OO 

will later prove to be very useful in non-Gaussian integrals later in the book because 
we sometimes only need the dependence of the integral on the parameter a. 
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Make the integral in eq. (3.58) dimensionless by the transformation y = x^/a. 


1 

G=^ e-y"dy 

Va J-oo 


(3.59) 


Note that the dependence on the parameter a appears as a simple factor in front of 
the integral. 

To evaluate the dimensionless integral, we square it and recast the product as a 
two-dimensional integral. 


G2 


/ OO -I pOO 

e~^ dx—^ / dy 

-OO \Oj J 


' —OO 

^oo poo 


i poo poo 

- / e”*-® ^dxdy 

^ J — OO j — OO 

a Jo 


— OO J —OO 
OO 

— r 


e ^ 2'Kr dr 


—e 


Jo a 


(3.60) 


The value of a Gaussian integral is therefore: 



(3.61) 


3.11 Stirling's Approximation for Nl 

As mentioned above, a difficulty in using the binomial distribution is that A^! becomes 
enormously large when N is even moderately large. For N = 25, Nl « 1.6 x 10^®, and 
we need to consider values of N of 10^° and higher! We would also like to differentiate 
and integrate expressions for the probability distribution, which is inconvenient in the 
product representation. 

The problem is solved by Stirling’s approximation, which is valid for large 
numbers—exactly the case in which we are interested. We will discuss various levels 
of Stirling’s approximation, beginning with the simplest. 

3.11.1 The Simplest Version of Stirling’s Approximation 

Consider approximating In A! by an integral. 

/ N \ N N 

In A! = In I j = ^ ln(n) « / ln(a;)cij: = A In A — A -|- 1 (3.62) 

\n=l / n=i 

This is equivalent to the approximation 

A! « A^exp(l - A) 


(3.63) 
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3.11.2 A Better Version of Stirling’s Approximation 

A better approximation can be obtained from an exact integral representation of N\. 

nOO 

m= e-^x^dx (3.64) 

Jo 

The correctness of eq. (3.64) can be shown by induction. It is clearly true for N = 0, 
since 0! = 1 = e~^dx. If it is true for a value A, then using integration by parts 
we can then prove that it is true for A + 1. 


pOQ 

{N+l)l= / e-^x^+^dx 

Jq 

poo 

= - / {-e-^{N + l)x^) dx 

Jo 

= (A + 1)A! 


(3.65) 


The integral in eq. (3.64) can be approximated by noting that the integrand is 
sharply peaked for large values of A. This suggests using the method of steepest 
descent, which involves approximating the integrand by a Gaussian function of the 
form 


g{x) = A exp 


{x - Xo)^ 
2a2 


(3.66) 


where A, Xo, and a are constants that must be evaluated. 

We can find the location of the maximum of the integrand in eq. (3.64) by setting 
the first derivative of its logarithm equal to zero. For the Gaussian function, this gives 


d 

dx 


Ing(x) 


d 

dx 


InA- 


{x 


- XoY 

2a2 


{x - Xo) ^ 

a2 


(3.67) 


or X = Xo- 

Gomparing eq. (3.67) to the first derivative of the logarithm of the integrand in 
eq. (3.64) equal to zero, we find the location Xo of the maximum 


d , , A , , 

0 = — —a; + Aina; = —IH- (3.68) 

dx X 

or X = Xo = N. The value of the amplitude is then just the integrand in eq. (3.64), 
g-Xj.N evaluated at a: = a:o = A, or A = e~^N^. 

The second derivative of the logarithm of a Gaussian function tells us the value of 
the variance. 


da; 2 


lng(a;) 


d (x — Xo) 
dx 0-2 



(3.69) 


When using this method of evaluating the variance of a Gaussian approximation, 
the second derivative of the logarithm of the function being approximated should be 
evaluated at Xo- 
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Table 3.4 Comparison of different levels of Stirling’s approximation with exact results for 


m 

eq. 

’Stirling (simple)’ refers to eq. 
(3.73). 

(3.63), 

‘Stirling (improved)’ to eq. 

(3.72), and 

‘Gosper’ to 

N 

N! 

Stirling 

(simple) 

error 

Stirling 

(improved) 

error 

Stirling 

(Gosper) 

error 

1 

1 

1 

0% 

0.922 

-7.79% 

0.996 

-0.398% 

2 

2 

1.47 

-26% 

1.919 

-4.05% 

1.997 

-0.132% 

3 

6 

3.65 

-39% 

5.836 

-2.73% 

5.996 

-0.064% 

4 

24 

12.75 

-47% 

23.506 

-2.06% 

23.991 

-0.038% 

5 

120 

57.24 

-52% 

118.019 

-1.65% 

119.970 

-0.025% 

10 

3628800 

1234098 

-66% 

3598694 

-0.83% 

3628559 

-0.007% 

20 

2.43 X lO^® 

5.87 X 10^^ 

-76% 

2.43 X lO^® 

-0.42% 

2.43 X 10^® 

-0.002% 


To find the value of take the second derivative of the logarithm of the integrand 
in eq. (3.64). 




iV, 


N 




(3.70) 


or 


= -^ = iV 
N 


(3.71) 


We can now use the formula for a Gaussian integral derived in Section 3.10. 

(x — iV)^"' 


poo pQ 

Nl= e-^x^dx « / 
Jo Jo 


e-^N^ exp 


2N 


dx « e-^N^V2TrN (3.72) 


As can be seen in Table 3.4, eq. (3.72) is a considerable improvement over eq. (3.63). 


The procedure used in this section to approximate a sharply peaked function by a 
Gaussian is extremely useful in statistical mechanics because a great many functions 
encountered are of this form. 


3.11.3 Gosper’s Approximation 

Finally, we mention a very interesting variant of Stirling’s approximation due to 
Gosper.^ 

(3.73) 



^Cited in http://mathworld.wolfram.com/StirlingsApproximation.html 
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Table 3.4 shows that Gosper’s approximation is extremely good, even for very small 
values of N. 


3.11.4 Using Stirling’s Approximation 

In applications to statistical mechanics, we are most often interested in the logarithm 
of the probability distribution, and consequently in In A!. This has the curious 
consequence that the simplest form of Stirling’s approximation is by far the most 
useful, even though Table 3.4 shows that its accuracy for Nl is very poor for large N. 
From Gosper’s approximation, we have a very accurate equation for In A!. 


In A! 


AlnA- A + - In 


2N+-]tt 


(3.74) 


In statistical mechanics we are most interested in large values of A, ranging from 
perhaps 10^^ to 10^"^. The relative error in using the simplest version of Stirling’s 
approximation, lnA!«AlnA —A, is of the order of 1/A, which is completely 
negligible even if A is as ‘small’ as 10^^. For A = 10^"^, In A! « Ain A — A is one 
of the most accurate approximations in physics. 


3.12 Binomial Distribution with Stirling's Approximation 

Since the last term in eq. (3.74) is negligible for large values of A, the most common 
approximation is to keep only the first two terms. If we use this form of Stirling’s 
formula, we can write the binomial distribution in eq. (3.47) in an approximate but 
highly accurate form. 


lnP(n|A) ss AlnA — nlnn— (A — n) ln(A — n) (3.76) 

+ n\np + (A — n) ln(l — p) 

Note that the contributions from the second term in Stirling’s approximation in 
eq. (3.74) cancel in eq. (3.76). 

Using Stirling’s approximation to the binomial distribution turns out to have a 
number of pleasant features. We know that the binomial distribution will be peaked, 
and that its relative width will be small. We can use Stirling’s approximation to find the 
location of the peak by treating n as a continuous variable and setting the derivative 
of the logarithm of the probability distribution with respect to n in eq. (3.47) equal 
to zero. 


— lnP(n|A) = 0 
on 


(3.77) 





Problems 33 


or 

d 

0 = — [In A^! — Inn! — ln(iV — n)! + nlnp + — nln(l — p)] 

d 

= — [N\\iN — N — n\n.n — n — {N — n) ln(A^ — n) — {N — n) 

+ nhip + N — nln(l — p)] 

= — In n + ln(A^ — n) + Inps — ln(l — p) (3.78) 

This equation determines the location Uq of the maximum probability from the 
equation: 


which has the solution 


rip _ p 

N — Up I — p 


(3.79) 


Up = pN (3.80) 

This means that the location of the peak, within the simplest of Stirling’s approxima¬ 
tions, gives the exact value of (n) = pN. 


3.13 Problems 


Sample program to simulate rolling a single die 

The computer program supplied first asks you how many times you want to roll a 
single die. It then chooses a random integer between 1 and 6, records its value, and 
repeats the process until it has chosen the number of random integers you specified. 
It then prints out the number of times each integer occurred and quits. 

The following VPython code is for the program OneDie_PARTIAL.py. 

from visual import * 

from visual, graph import * 

import random 

import sys 

from types import * 

from time import clock , time 

trials = 100 

print ( ” ^Number^of ^ t ri als , trials) 


sides = 6 
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histogram = zeros ( sides , int ) 
print ( histogram ) 

sum = 0.0 

j=0 

r=0 

while j < trials: 

r = int (random . random () * sides ) 
histogram [r] = histogram [r] + 1 

j=j+l 

j=0 

while j < sides : 

print ( histogram [ j ] , histogram[j] — t r i als / sides ) 

j=j+l 

Problem 3.1 

Rolling a die with a computer 

For this problem, either modify the VPython program (OneDie_PARTIAL.py) sup¬ 
plied, or write your own from scratch, using whatever computer language you 
prefer. 

1. Write a program (or modify the one given) to calculate and print out the 
histogram of the number of times each side occurs, the deviation of this number 
from one-sixth of the number of trials, the frequency with which each side occurs, 
and the deviation of this from one sixth. Hand in a copy of the code you used. 

2. Show a typical print-out of your program. 

3. Run the program for various numbers of random integers, starting with a small 
number, say 10, and increasing to a substantially larger number. The only upper 
limit is that the program should not take more than about one second to run. 
([Your time is valuable.) 

THIS IS A COMPUTATIONAL PROBLEM. 

ANSWERS MUST BE ACCOMPANIED BY DATA. 

Please hand in hard copies for all data to which you refer in your answers. 

4. As the number of trials increases, does the magnitude (absolute value) of the 
differences between the number of times a given side occurs and one-sixth of the 
number of trials increase or decrease? 

(Hint: This is not the same question as the next one.) 

5. As you increase the number of trials, does the ratio of the number of times each 
side occurs to the total number of trials approach closer to 1/6? 
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Problem 3.2 
Mutual independence 

We defined the concept of pairwise independence in this chapter. There is a related 
concept called mutual independence. Consider the set of random variables 

{A,\j = l,...,N} 

They are said to be mutually independent if for any subset of {Aj} containing n of 
these random variables, the marginal distribution satisfies the condition 

5 5 • ■ • ; ) — Pi ^Pk{^k ) ’ ’ ' Pni.^ri) 

Obviously, mutual independence implies pairwise independence. The question is 
whether pairwise independence implies mutual independence. 

Provide a proof or a counter-example. 

Problem 3.3 

A die with an arbitrary number of sides 

Suppose we have a die with S sides, where S is an integer, but not necessarily equal 
to 6 . The set of possible outcomes is then {n|n = 1,..., S'} (or {n|n = 0,..., S — 1}, 
your choice). Assume that all sides of the die are equally probable, so that P{n) = 1/S. 

Since this is partly a computational problem, be sure to support your answers with 
data from your computer simulations. 

1. What are the theoretical values of the mean, variance, and standard deviation as 
functions of SI The answers should be in closed form, rather than expressed as 
a sum (Hint: It might be helpful to review the formulas for the sums of integers 
and squares of integers.) 

2. Modify the program you used for an earlier problem to simulate rolling a die with 
S sides, and print out the mean, variance, and standard deviation for a number 
of trials. Have the program print out the theoretical predictions in each case, as 
well as the deviations from theory to facilitate comparisons. 

3. Run your program for two different values of S. Are the results for the mean, 
variance, and standard deviation consistent with your predictions? 

(Do not use such long runs that the program takes more than about a sec¬ 
ond to run. It would be a waste of your time to wait for the program to 
finish.) 

4. Experiment with different numbers of trials. How many trials do you need to 
obtain a rough estimate for the values of the mean and standard deviation? 
How many trials do you need to obtain an error of less than 1%? Do you need 
the same number of trials to obtain 1% accuracy for the mean and standard 
deviation? 
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Problem 3.4 

Independence and correlation functions 

We have shown that if the random variables A and B were independent and F{A) 
and G{B) were numerical functions defined on A and B, then 

{F{A)G{B)) = {F{A)){G{B)) 

Suppose have two random numbers, X and Y, and we know that: 

(XY) = {X){Y) 

Does that imply that X and Y are independent? 

Provide a proof or counter-example. 


Problem 3.5 

Some probability calculations (the Chevalier de Mere’s problem) 

1. Suppose we roll an honest die ten times. What is the probability of not finding 
a ‘3’ on any of the rolls. 

2. Calculate the following probabilities and decide which is greater. 

The probability of finding at least one ‘6’ on four rolls is greater than the probability 
of finding at least one ‘double 6’ on 24 rolls of two dice. 

Historical note: 

This is a famous problem, which is attributed to the French writer Antoine Gom- 
baud (1607-1684), who called himself the Chevalier de Mere (although according 
to Wikipedia he was not a nobleman). He was an avid but not terribly successful 
gambler, and he enlisted his friend Blaise Pascal (French mathematician, 1623-1662) 
to calculate the correct odds to bet on the dice rolls, described above. Pascal’s solution 
was one of the early triumphs of probability theory. 


Problem 3.6 
Generalized dice 

1. Modify your computer program to simulate the roll of N dice. Your program 
should let the dice have any number of sides, but the same number of sides for 
each die. The number of dice and the number of sides should be read in at the 
start of the program. 

One trial will consist of N rolls. Your program should calculate the sum of 
the N numbers that occur during each trial. It should also compare the results 
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for the mean, variance, and standard deviation of that sum with the theoretical 
predictions. 

2. Test the calculations that we have carried out for the mean, variance, and 
standard deviation of the sum of the numbers on the dice. In each case, obtain 
data for a couple of different run-lengths. 

Investigate the cases listed below. 

(a) Two dice, each with ten sides. 

(b) Ten dice, each with twenty sides. 

3. Use your program to investigate the width of the distribution for various numbers 
of two-sided dice. Does the width of the distribution increase or decrease with 
increasing numbers of dice? Do your results agree with the theory? 

Problem 3.7 
Mismatched dice 

In class, we derived the probability distribution for the sum of two dice using Kronecker 
deltas, where each die had S sides. 

For this problem, calculate the probability distribution for the sum of two dice 
using Kronecker deltas, when one die has four sides, and the other has six sides. 

Problem 3.8 

Computer simulations of mismatched dice 

1. Write a computer program to compute the probability distribution for the sum 
of two dice when each die has an arbitrary number of sides. 

Run your program for dice of four and six sides. 

2. Modify the computer program you wrote for the previous problem to compute the 
probability distribution for the sum of three dice when each die has an arbitrary 
number of sides. 

Run your program once for all dice having six sides, and once for any 
combination you think is interesting. 

Problem 3.9 
Sums of Os and Is 

The following questions are directly relevant to the distribution of ideal-gas particles 
between two boxes of volume Va and Ub. If U = Va + Vb, then p = VaIV and 1 — p = 
VbIV. 

1. Prove the following identity for binomial coefficients. 



N — n 


n + \ 
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2. Modify a new copy of your program to simulate the addition of an arbitrary 
number of independent random numbers, {nj\j = 1,... ,N}, each of which can 
take on the value 1 with probability P(l) = p and 0 with probability P{0) = 1 — p. 

Have the program print the histogram of values generated. (To save paper, 
print only the non-zero parts of the histograms.) 

Include a calculation of the theoretical probability from the binomial distri¬ 
bution, using the identity you proved at the beginning of this assignment. Have 
your program calculate the mean, variance, and standard deviation, and compare 
them to the theoretical values that we have calculated in class. 

Have the program plot the theoretical prediction for the histogram on the 
same graph as your histogram. Plot the deviations of the simulation data from 
the theoretical predictions on the second graph. (A discussion of how to make 
plots in VPython is included in Section 4 of the Appendix.) 

3. Run your program for the following cases, using a reasonable number of trials. 
Comment on the agreement between the theory and your computer experiment. 

1. 7V = 10, p = 0.5 

2. 7V = 30, p = 0.85 

3. iV = 150, p= 0.03 

Problem 3.10 

Sums of Os and Is revisited: the Gaussian approximation 

1. Modify the program you used for the simulation of sums of Os and Is to include 
a calculation of a Gaussian approximation to the binomial distribution. 

Keep the curve in the first graph that shows the theoretical binomial proba¬ 
bilities, but add a curve representing the Gaussian approximation in a contrasting 
color. 

Modify the second graph in your program to plot the difference between the 
full binomial approximation and the Gaussian approximation. 

2. Run simulations for various values of the probability p and various numbers of 
‘dice’, with a sufficient number of trials to obtain reasonable accuracy. (Remember 
not to run it so long that it wastes your time.) 

Comment on the accuracy (or lack thereof) of the Gaussian approximation. 

3. Run your program for the following cases, using a reasonable number of trials. 
Comment on the agreement between the theory and your computer experiment. 

1. iV= 10, p = 0.5 

2. N = 30, p = 0.85 

3. N = 150, p= 0.03 

Problem 3.11 

The Poisson distribution 

We have derived the binomial distribution for the probability of finding n particles 
in a subvolume out of a total of N particles in the full system. We assumed that the 
probabilities for each particle were independent and equal to p = VaIV. 
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When you have a very large number N of particles with a very small probability p, 
you can simplify this expression in the limit that p ^ 0 and TV —> oo, with the product 
fixed at pN = p. The answer is the Poisson distribution. 

-PmH = AM”exp(-p) 

n\ 

Derive the Poisson distribution from the binomial distribution. 

(Note that Stirling’s formula is an approximation, and should not be used in a 
proof.) 

Calculate the mean, variance, and standard deviation for the Poisson distribution 
as functions of p. 


Problem 3.12 

Numerical evaluation of the Poisson distribution 

In an earlier problem you derived the Poisson distribution. 

-PmH = ^M"exp(-p) 

1. Modify (a copy of) your program to read in the values of p and N and calculate 
the value of the probability p. Include an extra column in your output that gives 
the theoretical probability based on the Poisson distribution. 

(Note: It might be quite important to suppress rows in which the histogram is 
zero. Otherwise, the print-out could get out of hand for large values of iV.) 

2. Run your program for various values of p and N. How large does N have to be 
for the agreement to be good? 


4 

The Classical Ideal Gas: 
Configurational Entropy 


S = klogW 

Inscription on Boltzmann’s tombstone 
(first written in this form by Max Planck in 1900) 


This chapter begins the derivation of the entropy of the classical ideal gas, as outlined 
in Chapter 2. The first step is to separate the calculation of the entropy into two parts: 
one for the contributions of the positions of the particles, and one for the contributions 
of their momenta. In an ideal gas, these are assumed to be independent. As we will 
see, the total entropy is then just the sum of the two contributions. 

The contributions of the probability distribution for the positions of the particles, 
which we will call the configurational entropy, will be calculated in this chapter. 

4.1 Separation of Entropy into Two Parts 

The assumption that the positions and momenta of the particles in an ideal gas are 
independent allows us to consider each separately. 

As we saw in Section 3.3, the independence of the positions and momenta means 
that their joint probability can be expressed as a product of functions of q and p 
alone. 


P{q,p) = Pg{q)Pp{p) (4.1) 

According to Boltzmann’s 1877 definition, the entropy is proportional to the 
logarithm of the probability, to within additive and multiplicative constants. Since 
eq. (4.1) shows that the probability distribution in phase space can be expressed as 
a product, the total entropy will be expressed as a sum of the contributions of the 
positions and the momenta. 

The probability distribution in configuration space, Pq{q), depends only on the 
volume V and the number of particles, N. Consequently, the configurational entropy, 
Sq, depends only on V and IV; that is, Sq = Sq{V,N). 

The probability distribution in momentum space, Pp{p), depends only on the 
total energy, E, and the number of particles, N. Consequently, the energy-dependent 
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contribution to the entropy from the momenta, Sp, depends only on E and iV; that 
is, Sp = Sp{E,N). 

The total entropy of the ideal gas is given by the sum of the configurational and 
the energy-dependent terms. 

StotaiiE, y, N) = Sg{V, N) + Sp{E, N) (4.2) 

The thermodynamic quantities E, V, and N are referred to as ‘extensive’ para¬ 
meters (or observables, or variables) because they measure the amount, or extent, of 
something. They are to be contrasted with ‘intensive’ parameters, such as temperature 
or pressure, which do not automatically become bigger for bigger systems. 

4.2 Distribution of Particles between Two Subsystems 

Let us consider a composite system consisting of two boxes (or subsystems) containing 
a total of N distinguishable, non-interacting particles. We will name the boxes A 
and B, with volumes Va and Vb- The total volume is V = Va + Vb- The number of 
particles in A is Na, with Nb = N — Na being the number of particles in B. 

We can either constrain the number of particles in each box to be fixed, or allow 
the numbers to fluctuate by making a hole in the wall that separates the boxes. The 
total number of particles N is constant in either case. 

We are interested in the probability distribution for the number of particles in each 
subsystem after the constraint (impenetrable wall) is released (by removing the wall 
or making a hole in it). 

In keeping with our intention of making the simplest reasonable assumptions about 
the probability distributions of the configurations, we will assume that the positions 
of the particles are not only independent of the momenta, but are also mutually 
independent of each other. The probability density Pq{q) can then be written as a 
product. 


N 

P^{q) = P^{{r-}) = l[P,{r-) (4.3) 

f=i 

If we further assume that a given particle is equally likely to be anywhere in the 
composite system, the probability of it being in subsystem A is Va/V} 

Remember that this is an assumption that could, in principle, be tested with 
repeated experiments. It is not necessarily true. However, we are strongly prejudiced 
in this matter. If we were to carry out repeated experiments and find that a particular 
particle was almost always in subsystem H, we would probably conclude that there is 
something about the system that breaks the symmetry. 

The assumption that everything we do not know is equally probable—subject to 
the constraints—is the simplest assumption we can make. It is the starting point for 
all statistical mechanics. Fortunately, it provides extremely good predictions. 


^This can be shown more formally after the discussion of continuous random variables in Chapter 5. 
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If there are N particles that are free to go back and forth between the two 
subsystems, the probability distribution of iVyi is given by the binomial distribution, 
eq. (3.47), as discussed in Section 3.8. 


P{Na\N) 


Na!(N-Na)! (vJ V [vJJ 





N-Na 


(4.4) 

(4.5) 


To emphasize the equal standing of the two subsystems, it is often useful to write this 
equation in the symmetric form: 


P{Na,Nb) 


N\ /VaY" (VY""" 
NaINbI \ v J \v J ’ 


(4.6) 


with the constraint that Na + Nb = N. 


4.3 Consequences of the Binomial Distribution 

As shown in Section 3.7, the average value of Na from the binomial distribution is 


(NA = iV (^) 

By symmetry, we also have 

(NA = N (tl) 

SO that 

(Na) ^ W ^ N 

14 Vb y 


(4.7) 


(4.8) 


(4.9) 


The width of the probability distribution for Na is given by the standard deviation. 


6Na 



(4.10) 


By a fortunate quirk in the mathematics, the average value (Na) has exactly the 
same value as the location of the maximum of the approximate probability density 
using Stirling’s approximation, P{Na,N — Na)- This turns out to be extremely 
convenient. 
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4.4 Actual Number versus Average Number 

It is important to make a clear distinction between the actual number of particles 
at any given time and the average number of particles (Na)- 

The actual number of particles, Na, is a property of the system. It is an integer, 
and it fluctuates with time. 

The average number of particles, (Na), is part of a description of the system and 
not a property of the system itself. It is not an integer, and is time-independent. 

The magnitude of the fluctuations of the actual number of particles is given by 
the standard deviation, SNa, which is of the order of {Na) from eq. (4.10). If there 
are about 10^° particles in subsystem A, the actual number of particles Na will fluc¬ 
tuate around the value {Na) by about SNa ~ 10^° particles. The numerical difference 
between Na and (Na) is very big—and it becomes even bigger for bigger systems! 

Given the large difference between Na and (Na), why is {Na) at all useful? 
The answer lies in the fact that macroscopic measurements do not count individual 
molecules. The typical method used to measure the number of molecules is to weigh the 
sample and divide by the weight of a molecule. The weight is measured experimentally 
with some relative error—usually between 1% and one part in 10^. Consequently, 
using {Na) as a description of the system is good whenever the relative width of the 
probability distribution, 5Na/{Na), is small. 

From eq. (4.7), the relative width of the probability distribution is given by 


SNa 

{Na) 


{Na) 


{Na) 


Vb 

V 


1 1/2 



(4.11) 


The relative width is proportional to 1/y^ {Na), which becomes very small for a 
macroscopic system. For 10^® particles, the relative uncertainty in the probability dis¬ 
tribution is about 10”^°, which is much smaller than the accuracy of thermodynamic 
experiments. 

In the nineteenth century, as thermodynamics was being developed, the atomic 
hypothesis was far from being accepted. Thermodynamics was formulated in terms of 
the mass of a sample, rather than the number of molecules—which in any case, many 
scientists did not believe in. Fluctuations were not seen experimentally, so scientists 
did not make a distinction between the average mass in a subsystem and the actual 
mass. 

Maintaining the distinction between Na and {Na) is made more difficult by the 
common tendency to use a notation that obscures the difference. In thermodynamics, 
it would be more exact to use {Na) and similar expressions for the energy and 
other observable quantities. However, it would be tiresome to continually include 
the brackets in all equations, and the brackets are invariably dropped. We will also 
follow this practice in the chapters on thermodynamics, although we will maintain the 
distinction in discussing statistical mechanics. 

Fortunately, it is fairly easy to remember the distinction between the actual energy 
or number of particles in a subsystem and the average values, once the distinction has 
been recognized. 
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4.5 The ‘Thermodynamic Limit’ 

It is sometimes said that thermodynamics is only valid in the limit of infinite system 
size. This is implied by the standard terminology, which defines the ‘thermodynamic 
limit’ as the limit of infinite size while holding the ratio of the number of particles to 
the volume fixed. 

An obvious difficulty with such a point of view is that we only carry out experiments 
on finite systems, which would imply that thermodynamics could never apply to the 
real world. 

Another difficulty with restricting thermodynamics to infinite systems is that the 
finite-size effects due to containers, surfaces, and interfaces are lost. 

The point of view taken in this book is that thermodynamics is valid in the real 
world when the uncertainties due to statistical fluctuations are much smaller than the 
experimental errors in measured quantities. For macroscopic systems containing 10^° 
or more molecules, this means that the statistical uncertainties are of the order 10“^° 
or less, which is several orders of magnitude smaller than typical experimental errors. 
Even if we consider colloids with about 10^^ particles, the statistical uncertainties are 
about 10“®, which is still smaller than most measurements. 

Taking the limit of infinite system size can be a very useful mathematical approx¬ 
imation, especially when studying phase transitions, but it is not essential to either 
understand or apply thermodynamics. 


4.6 Probability and Entropy 

We have seen that the equilibrium value of {Na) is determined by the maximum of 
the probability distribution given by 


P{Na,Nb) 


N\ (VaY^ (Ye.Y" 
NaINbI {v ) {v J 


(4.12) 


with the constraint that Na + Nb = N. We can show the dependence of eq. (4.12) on 
the composite nature of the total system by introducing a new function: 


n, {N,v) 


yN 


(4.13) 


This allows us to rewrite eq. (4.12) as 


P{Na,Nb) 


{NA,VA)n, {Nb,Vb) 
fig {N,V) 


(4.14) 


At this point we are ready to make an extremely important observation. Since 
the logarithm is a monotonic function of its argument, the maximum of P {Na-, Nb) 
and the maximum of In [P {Na, Nb)] occur at the same values of Na and Nb- Using 
In [P {Na, Nb)] turns out to be much more convenient than using P {Na, Nb), partly 
because we can divide it naturally into three distinct terms. 
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ln[P(iV^,7VB)] =ln 


NaI 


■ In 




Nb 


Nb^- 


— In 




TV! 


(4.15) 


= In fig {Na, Va) + In fig {Nb,Vb) - In fig (TV, V) 


The first term on the right (in both forms of this equation) depends only on the 
variables for subsystem A, the second depends only on the variables for subsystem B, 
and the third term depends only on the variables for the total composite system. 

It will also be convenient to define a function 


N 


Sg{N,V) = kln(^^j+kXN 
= fc In fig {N, V) + kXN 


(4.16) 

(4.17) 


where k and X are both (at this point) arbitrary constants. The maximum of the 
function 


Sg,tot {Na, Nb,Vb) = kln[P {Na, TVs)] + 5, (TV, V) (4.18) 

= Sg{NA,VA) + Sg{NB,VB) 

with the usual constraint that Na + Nb = TV, then gives us the location of the 
equilibrium value of the average number of particles {Na}- 

We have seen in Section 3.8 that the width of the probability distribution is 
proportional to (Na)- Since the probability distribution is normalized, the value 
of its peak must be proportional to 1/y^ (Na)- This can also be seen from the gaussian 
approximation in eq. (3.57), with pN —> (Na), and 1 — p ^ VbIV. At the equilibrium 
values of Na and Nb, this gives 

InP(TV^,TVs)« -^ln{2n{NA){VB/V)) (4.19) 

Since the function 5^ (TV, y) is of order TV, and TV > {Na) >> In(TV^), the term 
fcln [P (TV^, TVs)] in eq. (4.18) is completely negligible at the equilibrium values of 
Na and Nb- Therefore, in equilibrium, we have 

Sg,tot {Na, Va, Nb,Vb) = Sg {Na, Va) + Sg {Nb,Vb) = Sg (TV, V) (4.20) 

We can now identify Sg^tot {Na,Va, Nb,Vb) as the part of the entropy of the 
composite system that is associated with the configurations; that is, with the positions 
of the particles. We will call Sg^tot {Na, Va, Nb,Vb) the total configurational entropy 
of the composite system. The functions Sg {Na,Va) and Sg {Nb,Vb) are called the 
configurational entropies of subsystems A and B. 

We have therefore found functions of the variables of each subsystem, such that 
the maximum of their sum yields the location of the equilibrium values 

Sg,tot {Na, ^4, Nb,Vb) = Sg {Na, Va) + Sg {Nb,Vb) (4.21) 

subject to the usual constraint that Na + Nb = TV. 
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The fact that the contributions to the configurational entropy from each subsystem 
are simply added to find the configurational entropy of the composite system is a very 
important property. It is traditionally known, reasonably enough, as ‘additivity’. On 
the other hand, the property might also be called ‘separability’, since we first derived 
the entropy of the composite system and were then able to separate the expression 
into the sum of individual contributions from each subsystem. 

The function that we have identified as the configurational entropy follows Boltz¬ 
mann’s definition (see Section 2.5) in being the logarithm of the probability (within 
additive and multiplicative constants). The fact that it is maximized at equilibrium 
agrees with Boltzmann’s intuition. This turns out to be the most important property 
of the entropy in thermodynamics, as we will see in Chapter 9. 

Boltzmann’s definition of the entropy always produces a function that has its 
maximum at equilibrium. Anticipating the discussion of the laws of thermodynamics 
in Part II, this property is equivalent to the Second Law of Thermodynamics, which 
we have therefore derived from statistical principles.^ 

4.7 An Analytic Approximation for the Configurational Entropy 

The expression for the configurational entropy becomes much more practical if we 
introduce Stirling’s approximation from Section 3.11. The entropy can then be dif¬ 
ferentiated and integrated with standard methods of calculus, as well as being much 
easier to deal with numerically. 

Because we are interested in large numbers of particles, only the simplest version 
of Stirling’s approximation is needed: In A^! ^ NlnN — N. The relative magnitude of 
the correction terms is only of the order of InN/N, which is completely negligible. It 
is rare to find an approximation quite this good, and we should enjoy it thoroughly. 

Using Stirling’s approximation, our expression for the configurational entropy 
becomes 



(4.22) 


where k and X are still arbitrary constants. In Chapter 8 we will discuss further 
constraints that give them specific values. 

This is our final result for the configurational entropy of the classical ideal gas. It 
will be supplemented in Chapter 6 with contributions from the momentum terms in 
the Hamiltonian. But first, we need to develop mathematical methods to deal with 
continuous random variables, which we will do in the next chapter. 


^There is still the matter of the nature of irreversibility, which will be addressed in Chapter 21. 
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Never try to walk across a river just because it has an average depth of four 
feet. 


Martin Friedman 


5.1 Continuous Dice and Probability Densities 

In Chapter 3 we discussed the basics of probability theory for discrete events. However, 
the components of the momenta of the particles in the ideal gas are continuous 
variables. This requires an extension of probability theory to deal with continuous 
random numbers, which we will develop in this chapter. 

To illustrate the essential features of continuous random numbers we will use a 
simple generalization of throwing a die. While a real die has six distinct sides, our 
continuous die can take on any real value, x, between 0 and 6. 

Assume that our continuous die is ‘honest’ in the sense that all real numbers 
between 0 and 6 are equally probable. Since there is an infinite number of possibilities, 
the probability of any particular number being found isp=l/oo = 0. 

On the other hand, if we ask for the probability of the continuous random number 
X being in some interval, the probability can be non-zero. If all values of x are equally 
probable, we would assume that the probability of finding x in the interval [a, h] is 
proportional to the length of that interval. 





(5.1) 


where A is a normalization constant. Since the probability of x being somewhere in 
the interval [0,6] must be 1, 





(5.2) 


we have A= 1/6. 

This leads us to define a ‘probability density function’ or simply a ‘probability 
density’, P{x) = 1/6, to provide us with an easy way of computing the probability of 
finding x in the interval [a, 6]. 
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P{[a,b]) = f P{x)dx 

J a 



b — a 
6 


(5.3) 


I apologize for using P to denote both probabilities and probability densities, even 
though they are very different things. This confusing notation certainly does not 
help keep the distinction between the two concepts in mind. On the other hand, 
it is usually easy to tell which concept is intended. Since much of the literature 
assumes that the reader can figure it out, this is probably a good place to become 
accustomed to doing so. 


5.2 Probability Densities 

We can use the idea of a probability density to extend consideration to cases in which 
all values of a continuous random number are not equally likely. The only change is 
that P{x) is then no longer a constant. The probability of finding x in an interval is 
given by the integral over that interval. 

P{[a,b]) = j P{x)dx (5.4) 

J a 

Generalization to multiple dimensions is simply a matter of defining the probability 
density of a multidimensional function. If there are two continuous random numbers, 
X and j/, the probability density is P{x,y). 

There is one feature of probability densities that might not be clear from our 
example of continuous dice. In general, although probabilities are dimensionless, 
probability densities have units. For example, if the probability of finding a single 
classical particle is uniform in a box of volume V, the probability density is 

P{x,y,z) = ^ (5.5) 

and it has units of When the probability density is integrated over a sub-volume 

of the box, the result is a dimensionless probability, as expected. 

Because probability densities have units, there is no upper limit on their values. 
Their values must, of course, be positive. However, unlike probabilities, they do not 
have to be less than 1. In fact, it is even possible for a probability density to diverge 
at one or more points, as long as the integral over all values is 1. 

Like probabilities, probability densities must be normalized. If indicates the 
entire range over which the probability density is defined, then 

I P{x)dx = 1 (5.6) 

Jn 


with obvious extensions to multi-dimensional, continuous random numbers. 
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Marginal probabilities are defined in analogy to the definition for discrete numbers. 

/ OO 

P{x,y)dy (5.7) 

-OO 

Conditional probabilities can also be defined if Px{x) yf 0. The conditional proba¬ 
bility P(y\x) can be written as 

P(y\^) = ( 5 - 8 ) 

Px[X) 

In general, we have 

Pix,y) = P{x\y)Py{y) 

= P{y\x)Pxix) (5.9) 

Independence is also defined in analogy to discrete probability theory. Two con¬ 

tinuous random numbers are said to be independent if 


P{x,y) = Px{x)Py{y). 


(5.10) 


Using eq. (5.9), we can see that if x and y are independent random numbers and 
Py{y) yf 0, the conditional probability P{x\y) is independent of y. 


P{x\y) 


P{x,y) 

Pyiy) 


Px{x)Py{y) ^ 

Py{y) 


(5.11) 


The average of any function F{x) can be calculated by integrating over the 
probability density. 



F{x)P{x)dx 


(5.12) 


Averages, moments, and central moments are all defined as expected from discrete 
probability theory. 


Mean: 


n-th moment: 


n-th central moment: 


/ OO 

xP{x)dx 

-OO 

/ OO 

x^P{x)dx 

-OO 


/ OO 

(a; — (x))" P{x)dx 

-OO 


(5.13) 


(5.14) 


(5.15) 
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As for discrete probabilities, the variance is defined as the second central moment, 
and the standard deviation is the square root of the variance. 

A word of warning: 

Many books refer to continuous random numbers as if they were discrete. They 
will refer to the probability of finding a random number in a given interval as being 
proportional to the ‘number of points’ in that interval. This is entirely incorrect and 
can cause considerable confusion. The number of points in an interval is infinite. It 
is even the same infinity as the number of points in any other interval, in the sense 
that the points in any two intervals can be mapped one-to-one onto each other. 
This error seems to have arisen in the nineteenth century, before physicists were 
entirely comfortable working with continuous random numbers. Although we are 
now in the twenty-first century, the error seems to be quite persistent. I can only 
hope that the next generation of physicists will finally overcome it. 


5.3 Dirac Delta Functions 

In Chapter 3 we discussed finding the probability of the sum of two dice using 
Kronecker delta functions. In the next section we will carry out the analogous calcu¬ 
lation for two continuous dice. Although the problem of two continuous dice is highly 
artificial, it is useful because its mathematical structure will often be encountered in 
statistical mechanics. 

For these calculations we will introduce the Dirac delta function, named for its 
inventor, Paul Adrien Maurice Dirac (English physicist, 1902—1984). The Dirac delta 
function is an extension of the idea of a Kronecker delta to continuous functions. This 
approach is not usually found in textbooks on probability theory—which is a shame, 
because Dirac delta functions make calculations much easier. 

Dirac delta functions are widely used in quantum mechanics, so most physics 
students will have encountered them before reading this book. However, since we 
will need more properties of the delta function than are usually covered in courses 
in quantum mechanics, we will present a self-contained discussion in the following 
section. 

5.3.1 Definition of Delta Functions 

To provide a simple definition of a Dirac delta function, we will first consider an 
ordinary function that is non-zero only in the neighborhood of the origin.^ 



^An alternative way of defining the Dirac delta function uses Gaussian functions, which has advantages 


when discussing derivatives. The approach used in this section has been chosen for its simplicity. 
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Clearly, this function is normalized. 

5e{x)dx = 1 

) 

In fact, since 5f^{x) is only non-zero close to the origin, 

f 6^{x)dx = 1 

J a 

as long as a < —e and b > e. 

It is also obvious that the function is symmetric. 


5^{x) = 5e{-x) 


(5.16) 


(5.17) 


(5.18) 


If we consider the function 5e{cx), where c is a constant, the symmetry of the function 
means that the sign of c is irrelevant. 


5f^{cx) = 5e(|c|a;) (5.19) 

Note that the width of the function 5^{cx) is a factor of l/|c| times that of (5e(x), while 
the height of the function remains the same (l/2e). 


5.3.2 Integrals over Delta Functions 

We can integrate over 6e{cx) by defining a new variable y = cx. 



Se{y)dy/\c\ = 


(5.20) 


When the argument of the delta function has a zero at some value of a; yf 0, the 
integral is unaffected as long as the limits of the integral include the zero of the 
argument of the delta function. As long as a < d/c — e and b > d/c + e, we have 




Se{y)dy/\c\ 


1 


(5.21) 


The Dirac delta function, S{x), is defined as the limit of S^{x) as e goes to zero. 


Six) = lim SJx) 

e^O 


(5.22) 


The Dirac delta function is zero for all a; yf 0 and infinite for a; = 0, so that calling 
it a function greatly annoys many mathematicians. For this reason, mathematics 
books on probability theory rarely mention the delta function. Nevertheless, if you 
are willing to put up with a mathematician’s disapproval, the Dirac delta function 
can make solving problems much easier. 
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The integral of the Dirac delta function is also unity when the limits of integration 
include the location of the zero of the argument of the delta function. 


L 



(5.23) 


as long as a < 0 and b > 0, and zero otherwise. 

Similarly, when the argument of the delta function is (cx — d ), the value of the 
integral is 



(5.24) 


as long as a < d/c and b > d/c, and zero otherwise. 

5.3.3 Integral of f{x) times a Delta Function 

The Dirac delta function can be used to pick out the value of a continuous function 
at a particular point, just as the Kronecker delta does for a discrete function. This is 
one of its most useful properties. 

Consider a function f{x) that is analytic in some region that includes the point 
Xo- Then we can expand the function as a power series with a non-zero radius of 
convergence 



(5.25) 


where f^^\x) is the j-th derivative of f{x). 



(5.26) 


Consider the integral over the product of f{x) and 5^{x — Xo)- 





(5.27) 
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All terms with odd j vanish. Keeping only the even terms and defining n = j/2 for 
even values of j, we can write the integral as 



f{x)S^{x - Xo)dx = ^ 

n—Q 


1 

(2ri+l)! 




= f{Xo) + 

n—1 


1 

(2ri+l)! 




(5.28) 


When we take the limit of e ^ 0, the function, <5e(-) becomes a Dirac delta function, 
and the right-hand side of the equation is just f{xo)- 


f 


f{x)5{x - Xo)dx = f{xo) 


(5.29) 


We can generalize eq. (5.29) to include more general arguments of the delta 
function. First, if the argument is of the form c{x — Xo), the result is divided by |c|. 

[ fix)6 {c{x - Xo)) dx = (5.30) 

J—oo 1^1 

We can further generalize this equation to allow the delta function to have an 
arbitrary argument, g{x). If g{x) has zeros at the points {xj\j = 1,... ,n}, and g'{x) 
is the derivative of g{x), the integral becomes 


J-OO \9'{Xj) 


(5.31) 


This can be proved by expanding g(x) about each of its zeros. 

Common errors in working with Dirac delta functions: 

• Not including all zeros of g{x). 

• Including zeros of g(x) that are outside the region of integration. 

• Not including the absolute value of g'{x) in the denominator. 


5.4 Transformations of Continuous Random Variables 

We are often interested in functions of random variables, which are themselves random 
variables. As an example, consider the following problem. 

Given two continuous random variables, x and y, along with their joint probability 
density, P{x,y), we wish to find the probability density of a new random variable, s, 
that is some function of the original random variables, s = f{x, y). The formal solution 
can be written in terms of a Dirac delta function. 







54 Continuous Random Numbers 


/ OO nOO 

/ P{x,y)5{s- f{x,y))dxdy 

-OO j —OO 


(5.32) 


As was the case for the corresponding discrete random variables, the probability 
density of s is automatically normalized. 

/ OO nOO pOO pOO 

P{s)ds= / / P{x,y)6{s - f{x,y))dsdxdy (5.33) 

-OO J—ooJ—ooJ — oo 

/ OO p(X) 

/ P{x,y)dxdy=l 

-OO J —OO 

The last equality is due to the normalization of P[x,y). 


5.4.1 Sum of Two Random Numbers 


To see how eq. (5.32) works in practice, consider the sum of two uniformly distributed 
random numbers. We will use the example of two continuous dice, in which the two 
random numbers x and y each take on values in the interval [1,6], and the probability 
density is uniform. 


P{x,y) = 1/36 


(5.34) 


This probability density is consistent with the normalization condition in eq. (5.6). 
The probability density for s = x + y is then 


6 /.6 


Pis) = / / P{x,y)6{s-{x + y))dxdy 

Jo Jo 

I rG rG 


36 


d{s — {x + y))dx dy 


0 ./o 


(5.35) 


Following the same procedure as in Section 3.4, we first carry out the integral 
over y. 



x + y))dy 


10<s — x<6 
0 otherwise 


(5.36) 


In direct analogy to the procedure for calculating the sum of two discrete dice in 
Section 3.4, we have two conditions on the remaining integral over x. Only those values 
of X for which both x < s and x > s — 6 contribute to the final answer. These limits 
are in addition to the limits of x < 6 and x > 0 that are already explicit in the integral 
over X. Since all four of these inequalities must be satisfied, we must take the more 
restrictive of the inequalities in each case. Which inequality is the more restrictive 
depends on the value of s. Determining the proper limits on the integral over x is 
illustrated in Table 5.1, which should be compared to Table 3.3 for the corresponding 
discrete case. 

For s < 6, the lower bound on x is 0 and the upper bound is s. For s > 6, the 
lower bound on x is s — 6 and the upper bound is 6. The sums can then be evaluated 
explicitly. 
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Table 5.1 Determining the limits for the second integral when evaluating eq. (5.35). 



lower limit 

upper limit 

From limits on integral: 

a; > 0 


x < 6 

From delta function: 

X > s — 

6 

X < s 

More restrictive if s < 6 

X >0 


X < s 

More restrictive if s > 6 

X > s — 

6 

X < 6 


P{s) 



s 

12-s 
' 36 


s < 6 
s > 6 


(5.37) 


5.5 Bayes’ Theorem 

Bayes’ theorem, eq. (3.13), was derived in Section 3.3.1. 


P{A\B) 


P{B\A)Pa{A) 

Pb{B) 


(5.38) 


If we accept the Bayesian definition of probability in Section 3.1, we can apply 
eq. (5.38) to the determination of theoretical parameters from experiment. 

Let X denote the data from an experiment, while 9 denotes the theoretical 
parameter(s) we wish to determine. Using standard methods, we can calculate the 
conditional probability P{X\9) of observing a particular set of data if we know the 
parameters. However, that is not what we want. We have the results of the experiment 
and we want to calculate the parameters conditional on that data. 

Bayes’ theorem gives us the information we want about the theoretical parameters 
after the experiment. It provides the information on a particular set of measurements 
X, as the conditional probability for 9 in the following equation. 


P{9\X) 


P{X\9)Pg{9) 

Px{X) 


(5.39) 


In Bayesian terminology, Po{9) is called the ‘prior’ and represents whatever knowledge 
we had of the parameters before we carried out the experiment. The conditional prob¬ 
ability P{X\9) — viewed as a function of 9 —is called the ‘likelihood’. The conditional 
probability P{9\X) is known as the ‘posterior’, and represents our knowledge of the 
parameters after we have obtained data from an experiment. 

For example, suppose you want to determine the value of the asymptotic frequency 
/ (the frequentist’s probability) for a particular random event. Assume you have done 
an experiment with a binary outcome of success or failure, and you obtained n = 
341,557 successes in iV = 10® trials. The experimental data, denoted above by X, is 
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n in this case. You would immediately guess that / « 0.341557, but how close to that 
value is it? 

Our theoretical parameter, denoted by 0 in eq. (5.39), is the value of /. Suppose 
we knew nothing of the value of / before the experiment, except that 0 < / < 1. A 
reasonable description of our knowledge (or lack of it) would be to say that our prior is 
a uniform constant for values between zero and 1. From the normalization condition, 
the constant must be 1. 


fo /<0 

Pf{f) ={l0<f<l (5.40) 

[o />1 

The likelihood is given by the binomial distribution 

PN{n\f) = r(l - (5.41) 

n\{N — n)\ 

As is usual in Bayesian calculations, we will ignore PN,n{ti)\ since it does not depend 
on /, it simply plays the role of a normalization constant. Putting eqs. (5.39), (5.40), 
and (5.41) together and ignoring multiplicative constants, we have the probability 
density for /. 


Pjv(/|n)oc/"(l-/)'^-" (5.42) 

For large n and N, this is a sharply peaked function. The maximum can be found 
by setting the derivative of its logarithm equal to zero. 

d d 71 N — 71 

= — [nln f + {N - n)ln(l - /) + constants] = -- ^ ^ = 0 (5.43) 

The maximum is therefore located at 


71 

fmax = ^ (5.44) 

as expected, since we had n successes in N trials. 

The variance can be found by approximating the distribution by a Gaussian, finding 
the second derivative, and evaluating it at fmax- 


df_ 

dp 


[nln / + {N — n) ln(l — /)] 


d n N — n 

^U~ 1-/. 


n N — n 
P~f]PP]p 


(5.45) 

(5.46) 


Evaluating this derivative at / = fmax = n/N The variance is then given by 


1 


N 


N 


N 


I- frr 


fmax{^ fmax') 


(5.47) 
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or 


2 J max 
(J = 


fmaxiX fmax^ 


N 


The standard deviation is then 


(T = 



(1 fmax 

~N 


so that the uncertainty in the value of / 


is proportional to 1/VN. 


(5.48) 


(5.49) 


5.6 Problems 


Problem 5.1 
Probability densities 

1. Given a continuous random number x, with the probability density 

P{x) = Aexp(—2a;) 

for all X > 0, find the value of A and the probability that x > 1. 

2. It is claimed that the function 

P{y) = - 

V 

where B \s a. constant, is a valid probability density for 0 < y < 1. 
Do you agree? If so, what is the value of B1 

3. It is claimed that the function 



where C is a constant, is a valid probability distribution for 0 < y < 1. 

Do you agree? If so, what is the value of Cl 

4. The probability distribution P(x, y) has the form 

P(x,y) = Dxy 

for the two random numbers x and y in the region x > 0, y > 0, and x + y < 1, 
where D is a constant, and P{x,y) = 0 outside this range. 

(a) What is the value of the constant D1 

(b) What is the probability that x < 1/2? 

(c) Are X and y independent? 





58 Continuous Random Numbers 


Problem 5.2 

How well can we measure the asymptotic frequency? 

I have run a number of computer simulations in which a ‘success’ occurred with 
probability p. It would be no fun if I told you the value of p, but I will tell you 
the results of the simulations. 


Trials 

Successes 

10^ 

69 

10^ 

641 

10^ 

6353 

10*^ 

63738 

10® 

637102 

10" 

6366524 


1. Using Bayesian statistics, we showed that we can represent our knowledge of the 
value of p by a probability distribution (or probability density) P{p\n), which is 
the conditional probability density for p, given that an experiment of N trials 
produced n successes. P{p\n) is found using Bayes’ theorem. Note that P{n\p), 
the conditional probability of finding n successes in N trials when the probability 
is p, is just the binomial distribution that we have been studying. 

Find the location of the maximum of this probability density by taking the 
first derivative of P{p\n) and setting 

^ lnP(p|n) = 0 

Find the general expression for the width of P{p\n) by taking the second deriva¬ 
tive. The procedure is essentially the same as the one we used for calculating the 
Gaussian approximation to the integrand in deriving Stirling’s approximation. 

2. Using the result you obtained above, what can you learn about the value of p 
from each of these trials? Is the information from the various trials consistent? 
(Suggestion: A spreadsheet can be very helpful for calculations in this problem.) 


Problem 5.3 

Continuous probability distributions 

Determine the normalization constant and calculate the mean, variance, and standard 
deviation for the following random variables. 

1. For a; > 0 

P{x) = Aexp(—ax) 

2. For X > 1 


P(x) = Bx-^ 
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Problem 5.4 

Integrals over delta functions 

Evaluate the following integrals in closed form: 


1 . 


2 . 


/ OO 

x‘^5{x‘^ — y'^)dz 

-OO 


exp(—x)i5(sin(x)))c?x 


-1 


3. The probability density P{x,y) has the form 


P(x,y) = 2Axy 


for the two random numbers x and y in the range x > 0, y > 0, and a; + y < 1, 
and P{x,y) = 0 outside this range. 

What is the probability distribution for the following new random variable? 


z = X + y 


Problem 5.5 

Transforming random variables 

Consider the two random variables x and y. Their joint probability density is given by 

P{x,y) = A exp - 2y] 
where A is a constant, x > 0, y > 0, and y < 1. 

1. Evaluate the constant A. 

2. Are x and y independent? Justify your answer. 

3. If we define a third random number by the equation 

z = x^ + 2y 

what is the probability density P{z)l 


Problem 5.6 

Maxwell Boltzmann distribution 

In the near future we will derive the Maxwell-Boltzmann distribution for the velocities 
(or the momenta) of gas particles. The probability density for the x-component of the 
velocity is 


60 
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P{Va;) = A exp 


—[3-mv 


2 

X 


1. What is the value of the normalization constant A1 

2. What is the probability density for the velocity v = {va;,Vy,Vz)l 

3. What is the probability distribution (probability density) of the speed (magnitude 
of the velocity)? 


Problem 5.7 

Two one-dimensional, ideal-gas particles 

Consider two ideal-gas particles with masses rriA and rriB, confined to a one¬ 
dimensional box of length L. Assume that the particles are in thermal equilib¬ 
rium with each other, and that the total kinetic energy is if = Ea + Eb- Use the 
usual assumption that the probability is uniform in phase space, subject to the 
constraints. 

1. Calculate the probability distribution P{Ea) for the energy of one of the particles. 

2. Calculate the average energy of the particle, (Ea)- 

3. Find the most probable value of the energy Ea (the location of the maximum of 
the probability density). 


Problem 5.8 

Energy distribution of a free particle 

Suppose we have an ideal gas in a finite cubic box with sides of length L. The system 
is in contact with a thermal reservoir at temperature T. We have determined the 
momentum distribution of a single particle to be: 


P{p,z) 


X exp 



M!' 

2m 


Find P{E), where E is the energy of a single particle. 


Problem 5.9 

Particle in a gravitational field 

An ideal gas particle in a gravitational field has a probability distribution in momen¬ 
tum p and height z of the form 


P{p, z) 


X exp 



(5mgz 


for 0 < 2 < oo and all values of momentum. 
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1. Evaluate the constant X. 

2. Calculate the average value of the height: (z). 

3. Calculate the probability distribution of the total energy of the particle. 


6 

The Classical Ideal Gas: 
Energy-Dependence of Entropy 


For those who want some proof that physicists are human, the proof is in the 
idiocy of all the different units they use for measuring energy. 

Richard Feynman 


This chapter provides the second half of the derivation of the entropy of the classical 
ideal gas, as outlined in Chapter 2 and begun in Chapter 4. In the present chapter 
we will calculate the probability distribution for the energy of each subsystem. 
The logarithm of that probability then gives the energy-dependent contribution 
to the entropy of the classical ideal gas. The total entropy is just the sum of 
the configurational entropy and the energy-dependent terms, as discussed in Sec¬ 
tion 4.1. 


6.1 Distribution for the Energy between Two Subsystems 

We again consider the composite system that consists of the two subsystems discussed 
in Chapter 2. Subsystem A contains Na particles and subsystem B contains Nb 
particles, with a total oi N = Na + Nb particles in the composite system. Since we 
are dealing with classical, non-interacting particles, the energy of each subsystem is 
given by 


i=i 


( 6 . 1 ) 


where a = ^ or i? and m is the mass of a single particle. The momentum of the j-th 
particle in subsystem a is Pa,j. 

The composite system is perfectly isolated from the rest of the universe, and the 
total energy E = Ea + Eb is fixed. We are interested in the case in which the two 
subsystems can be brought into thermal contact to enable the subsystems to exchange 
energy, so that the composite system can come to thermal equilibrium. 

A partition that is impervious to particles, but allows energy to be exchanged 
between subsystems, is called a ‘diathermal’ wall, in contrast to an adiabatic wall that 
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prevents the exchange of either particles or energy. We wish to calculate the probability 
distribution of the energy between two subsystems separated by a diathermal wall. 

To determine the probability distribution of the total energy we must make an 
assumption about the probability distribution in momentum space. The simplest 
assumption is that the momentum distribution is constant, subject to the con¬ 
straints on the energy. This is the counterpart of the assumption made in Chap¬ 
ter 4, that the positions of the particles were uniformly distributed in configuration 
space. 

With the assumption of uniform probability density in momentum space, we can 
calculate the probability distribution for the energy from that of the momenta, using 
the methods from Section 5.4. We use Dirac delta functions to select states for which 
system A has energy Ea and system B has energy Eb- Conservation of energy in the 
composite system of course requires that Eb = E — Ea, but this form of writing the 
probability distribution will be useful in Section 6.2 when we separate the contributions 
of the two subsystems. 

Since the notation can become unwieldy at this point, we use a compact notation 
for the integrals 


OO OO 


J ■ ■ ■ J {■■■ )dp%i ■ ■ ■ dp%NA = J (■■■ )dPA 


( 6 . 2 ) 


— OO —OO 


and write: 


P{Ea,Eb) = 




OO f Nb |j 7„ -t 


dpB 


J SiE-Y. 


\pj\ 


(6.3) 


j=i 2m 


dp 


The integral over dp in the denominator goes over the momenta of all particles in the 
composite system. 

The denominator of eq. (6.3) is chosen to normalize the probability distribution. 


P{EA,E-EA)dEA = l 


By defining a function 


{Ea, 




dp. 


(6.4) 


(6.5) 


we can write eq. (6.3) in an even more compact form that highlights its similarity to 
eq. (4.14). 


{Ea, Na) fig {Eb,Nb) 
^E {E,N) 


P{Ea,Eb) 


( 6 . 6 ) 
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6.2 Evaluation of CIe 

To complete the derivation of P {Ea, Eb), we need to evaluate the function Ue {E, N) 
in eq. (6.5). Fortunately, this can be done exactly by taking advantage of the symmetry 
of the integrand to transform the 3iV-dimensional integral to a one-dimensional 
integral, and then relating the integral to a known function. 


6.2.1 Exploiting the Spherical Symmetry of the Integrand 

The delta function in eq. (6.5) makes the integrand vanish everywhere in momentum 
space, except on a sphere in momentum space defined by energy conservation. 

N 

2mE = J2\P,\^ ( 6 . 7 ) 

t=i 


(For simplicity, we have dropped the subscript a.) The radius of this 37V-dimensional 
sphere is clearly \/2mE. 

Since the area of an n-dimensional sphere is given hy A = where Sn is 

a constant that depends on the dimension, n, the expression for the function can 
be immediately transformed from a 3iV-dimensional integral to a one-dimensional 
integral. 


nE{E,N) =/•••/ <5 E-J2 

J J-OQ \ 

— OO \ •J 


\pj 


2m 


dpi--- dp3N 


= J S,,p^^-^5 (^E - 
0 



dp 


( 6 . 8 ) 


The new integration variable is just the radial distance in momentum space. 


P 


2 


N 




(6,9) 


To evaluate the integral in eq. (6.8), transform the variable of integration by 
defining x = /2m, which implies that p^ = 2mx and pdp = mdx. Inserting this in 

the expression for VIe {E, N), we can evaluate the integral. 


Qe {E, N) = Snj P^^-^5 (^E - dp 
0 


= Sn j{2mxY^^-^^/^S{E 
0 


x) mdx 


= Snm{2mE)^^^-^'>/^ 


( 6 . 10 ) 
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The only thing remaining is to evaluate Sn, which is the surface area of an n- 
dimensional sphere of unit radius. 

6.2.2 The Surface Area of an n-Dimensional Sphere 

The volume of a sphere in n dimensions is given by Cn’r'^-, where Cn is a constant. By 
differentiation, the area of the sphere is nCnr"”^, so that Sn = nCn- 

To evaluate Cn we can use a trick involving Gaussian integrals. We begin by taking 
the n-th power of both sides of eq. (3.61). 
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'OO 
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( 6 . 11 ) 


We can transform the final integral in eq. (6.11) to a more convenient form by changing 
the integration variable to t = 





( 6 . 12 ) 


The integral in eq. (6.12) is well known; it is another representation of the factorial 
function. 



(6.13) 


We can see why this is so by induction. Begin with m = 0. 



(6.14) 
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If eq. (6.13) is valid for m, then we can use integration by parts to prove that it is 
valid for (m +1). 



e"* t^+^dt 


poo 

[e-*{m + l)t^] “ - (to + 1) / (-e"‘) f 

Jo 

poo 

(to + 1 ) / t"^dt 

Jo 


dt 


(to + 1)to! 
(to + 1)! 


(6.15) 


This confirms the validity of eq. (6.13) for all positive integers. 

A convenient consequence of eq. (6.13) is that it provides an analytic continuation 
of to! to non-integer values. It is traditional to call this extension of the concept of 
factorials a Gamma (T) function. 


r(TO-I-1) = m! = / e ^f^dt (6.16) 

J —oo 


For many purposes, r(-) is a very useful notation. However, the shift between to and 
TO -|- 1 can be a mental hazard. I will stay with the factorial notation, even for non¬ 
integer values. 

Returning to eq. (6.12), we can now complete the derivation of C^. 


1 r°° 

2 Jo 

= C'„(^) (n/2-1)! 

= {nl2)\Cn (6.17) 

This gives us our answers for Cn and Sn = nCn- 


TT ' 

(n/2)! 

(6.18) 


\ul2)\ 

(6.19) 


6.2.3 Exact Expression for 

Inserting eq. (6.19) into the expression for VIe {E,N) in eq. (6.10) we find 


flEiE^N) 


3JY^3Ar/2 

(3iV/2)! 


m{2mE) 


(3Af-l)/2 


( 6 . 20 ) 
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An excellent approximation to the logarithm of flE{E,N) can be found using 
Stirling’s approximation. The errors are of order l/N. 


InflEiE^N) 



( 6 . 21 ) 


The constant X in eq. (6.21) can be calculated, but will not be needed at this point. 


6.3 Probability Distribution for Large N 

Putting the functional form of LIe into eq. (6.6)—and ignoring constants for the time 
being—we find the energy-dependence of the probability distribution. 


P{Ea,E-Ea) = 


ns iEA,NA)nEiE-EA,NB) 


LIe {E,N) 
cx {E - 


( 6 . 22 ) 


We are primarily interested in finding the average energy, {Ea), because the energy 
Ea has a very narrow probability distribution, and the measured energy will almost 
always be equal to (Ea) within experimental error. We want to find the width of the 
distribution, SEa, only to confirm that the distribution is narrow. 

The average, {Ea), is most easily found by approximating it by the location of 
the maximum probability, which, in turn, is found by taking the derivative of the 
logarithm of the energy distribution and setting it equal to zero. 
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lnP{EA,E-EA) 


fSNA-l 
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) E-Ea 


(6.23) 


Solving this equation gives us the location of the maximum of the probability distri¬ 
bution, which is located at (Ea) to very high accuracy. 


A^.max - {Ea} - ^^E. 

For a large number of particles, this becomes 


Ea 


,max — 



or 


£'A,max _ E _ {Ea) 
Na ~ N ~ Na 


(6.24) 


(6.25) 


(6.26) 


with a relative error of the order oi N When N = 10^°, or even N = this is 
certainly an excellent approximation. 
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Note that the energy per particle is the same in each subsystem. This is a special 
case of the equipartition theorem, which we will discuss in more detail when we 
come to the canonical ensemble in Chapter 19. 


For large N, the probability distribution is very sharply peaked. It can be approx¬ 
imated by a Gaussian, and we can use the same methods as we did in Section 3.9 to 
determine the appropriate parameters. In particular, we can calculate the width of the 
probability distribution by taking the second derivative of the logarithm, which gives 
us the negative reciprocal of the variance. 
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(6.27) 


Evaluating this at the maximum of the function, EA,max = ENa/N, and assuming 
that we are dealing with large numbers of particles, this becomes 
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The variance of Ea is then: 

^2 _ 2{Ea? (NaNb\ _ {Ea? (2Nb\ 
3Nl \ N ) N \3 Na) 


(6.29) 


The width of the energy distribution is then given by the standard deviation, 
SEa = (JEa- 


CTEa 


{Ea) 


[T (2Nb\ 

V N XiNAj 
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= {Ea) 
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1/2 


(6.30) 


From eq. (6.30), we see that since Ea oc Na, the width of the probability distrib¬ 
ution increases with ?Na, while the relative width. 


fir f2NBY^ 

{Ea) {Ea) y Na { 3N J 


(6.31) 


decreases with ^Na- 
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The behavior of the width and relative width is analogous to the ^/Na behavior 
we saw in eq. (4.10) for the width of the probability distribution for Na- As the size 
of the system increases, the width of the probability distribution for Ea increases, 
but the relative width decreases. For a system with 10^° particles, the typical relative 
deviation of Ea from it average value is of the order of which is a very small 

number. 

This increase of the standard deviation 6Ea with the size of the system has the 
same significance discussed in Section 4.4 for 6Na- The average value of the energy 
is a description of the subsystem that we find useful; it is not the same as the true 
energy of the subsystem at any given time. 

In analogy to the corresponding question in Section 4.4, we might ask the following. 
Given the large value of SEa, why is (Ea) at all useful? The reason is that the relative 
width of the probability distribution 


SEa 

(Ea) 





(6.32) 


becomes smaller as the size of the system increases. For 10^° particles, 5Ea/{Ea) is 
of the order of = 10“^°, which is much too small to be seen by macroscopic 

experiments. 

For both the energy and the number of particles in a subsystem, we have found 
that the relative width of the probability distribution is very narrow. As long as 
the width of the probability distribution is narrower than the uncertainty in the 
experimental measurements, we can regard the predictions of probability theory 
(statistical mechanics) as giving effectively deterministic values. 

It is important to note both that it is not necessary to take the ‘thermodynamic 
limit’ (infinite size) for thermodynamics to be valid, and that it is necessary for the 
system to be large enough so that the relative statistical fluctuations of measured 
quantities is negligible in comparison with the experimental error. 


6.4 The Logarithm of the Probability Distribution and the 
Energy-Dependent Terms in the Entropy 

In direct analogy to the derivation of eq. (4.18) for the configurational contributions 
in Chapter 4, we can find the the energy-dependent term in the entropy by taking the 
logarithm of eq. (6.6). 

In analogy to eq. (4.16), we can define a function to describe the energy-dependent 
contributions to the entropy of the classical ideal gas. 

SE,c. = k\xtnE{E^,No) (6.33) 

where a refers to one of the subsystems, A or B, or the composite system when a is 
omitted. The energy-dependent contributions to the entropy of the composite system 
are then: 



70 The Classicsl Ideal Gas: Energy-Dependence of Entropy 


SE,tot {Ea, Na, Eb,Nb) = klnP {Ea, Na, Eb,Nb) + Se {E, N) 

= Se {Ea, Na) + Se {Eb,Nb) 


(6.34) 


SE,tot {Ea, Na, Eb, Nb) is the energy-dependent contribution to the total entropy 
of the composite system, while Se {Ea,Na) and Se {Eb,Nb) as the corresponding 
energy-dependent contributions to the entropies of the subsystems. 

Eq. (6.34) shows that the energy-dependent contributions to the entropy of an 
ideal gas are additive, as the configurational terms were shown to be in Section 4.6. 

As was the case for the configurational entropy, the height of the probability 
distribution for Ea is proportional to \/Na, so that in equilibrium 


klnP{EA,NA,EB,NB) (x In N <m: Se {E,N) oc N 


(6.35) 


and the energy-dependent contributions to the entropy of the composite system in 
equilibrium is given by S's {E,N). 


SE,tot {Ea, Na, Eb, Nb) = Se {Ea, Na) + Se {Eb,Nb) = Se {N, V) (6.36) 


From eq. (6.21), we can see that for large N, the energy contributions of the entropy 
are given by 



Se{E,V) = kBlnnE{E,V) Ki kBN 


(6.37) 


-h A 


where the errors are of order 1/N, and X is a constant that we will not need to 
evaluate at this point. 
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Classical Gases: Ideal and Otherwise 


Even imperfection itself may have its ideal or perfect state. 
Thomas de Quincey 


In Chapter 2 we introduced Boltzmann’s 1877 definition of the entropy in terms of 
the logarithm of a probability. We assumed that the positions and momenta of the 
particles in a classical ideal gas were independent random variables, so that we could 
examine their contributions to the entropy independently. In Chapter 4 we calculated 
the contributions from the positions of the particles, and in Chapter 6 we calculated 
the contributions from the momenta of the particles. Now we are in a position to put 
them together to find the total entropy of the composite system. 

7.1 Entropy of a Composite System of Classical Ideal Gases 

We have seen that the contributions to the total entropy of the composite system 
from the positions of the particles in eq. (4.16) and their momenta in eq. (6.33) simply 
add to give the total entropy. Since both the configurational and energy-dependent 
contributions to the entropy of the composite system are separable into contributions 
from each subsystem, the total entropy is also separable. It is easier (and requires 
less space) to look at the expression for the entropy of a single system at this point, 
rather than the composite system needed for the derivations in Chapters 4 and 6. For 
simplicity we will omit the subscripts that indicate which subsystem we are referring 
to, since the mathematical form is the same in each case. 



yN 



(7.1) 


-I- In 


This equation contains two constants that are still arbitrary at this point: fc, which 
will be discussed in Chapter 8, and X', which we will discuss further below. 

For any macroscopic system—that is, any system with more than about 10^*^ 
particles—Stirling’s approximation is excellent. We can also replace 3N/2 — 1 with 
3iV/2, which has a relative error of only 3/2N. The result is the entropy of a classical 
ideal gas. 



(7.2) 
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The constant X = X' +1 comes from the replacement of N\ by Stirling’s 
approximation. 

The value of X in eq. (7.2) is completely arbitrary within classical mechanics. 
However, there is a traditional value chosen for X. 



47rm\ 

) 



(7.3) 


The constant h in this expression is Planck’s constant, taken from quantum mechanics. 
The presence of Planck’s constant in eq. (7.3) makes it obvious that this value of X has 
nothing to do with classical mechanics. It is determined by solving for the entropy of a 
quantum mechanical gas—for which the additive constant does have a meaning—and 
taking the classical limit. We will carry out this procedure in Chapters 26, 27, and 28. 


7.2 Equilibrium Conditions for the Ideal Gas 

By definition, the entropy of any composite system should be a maximum at equi¬ 
librium. Eq. (7.2) satisfies that condition with respect to both energy and particle 
number. We will show this explicitly below—first for equilibrium with respect to 
energy, and then with respect to particle number. 

7.2.1 Equilibrium with Respect to Energy 

We want to confirm that the expression we have derived for the entropy predicts the 
correct equilibrium values for the energies of two subsystems. To do that, consider 
an experiment on a composite system with fixed subvolumes, Va and Vb, containing 
Na and Nb particles respectively. The total energy of the particles in the composite 
system is E. There is a diathermal wall separating the two subvolumes, so that they 
can exchange energy. 

We want to find the maximum of the entropy to confirm that it occurs at the 
equilibrium values of Ea and Eb = E — Ea- We find the maximum in the usual way, 
by setting the partial derivative with respect to Ea equal to zero. 

—5(Ea, Pa, Na; E - Ea^Vb^Nb) = ^ [Sa{Ea, Pa, Na) + Sb{Eb,Vb, Nb)] 
atjA o±tA 

f) fiF' f) 

= r., NA + 

o o 

= ■w^Sa{Ea,Va,Na) — „ Sb{Eb^VbtNb) 
atjA ClltB 

= 0 (7.4) 

This gives us an equilibrium condition that will play a very important role in the 
development of thermodynamics. 

OSa OSb 


BEa BEb 


(7.5) 
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Since the partial derivative of the entropy with respect to energy is 


,,, a 

dE ~ ^ dE 


3 (E 


= \kN^[\nE-\nN] 
3kN 


2E 


the equilibrium condition is predicted to be 

Ea _ Eb 


In I — 




(7.6) 


(7.7) 


which is the equilibrium condition found in eq. (6.26). 


7.2.2 Equilibrium with Respect to the Number of Particles 

We now demonstrate that our expression for the entropy predicts the correct equilib¬ 
rium values of the numbers of particles in each subsystem. Consider an experiment on 
a composite system with fixed subvolumes, Va and Vb ■ The total number of particles 
is N, but there is a hole in the wall between the two subvolumes, so that they can 
exchange both energy and particles. The total energy of the particles in the composite 
system is E. 

Since the two subsystems can exchange particles, they can certainly exchange 
energy. The derivation in the previous subsection is still valid, so that the entropy is 
correctly maximized at the equilibrium value of the energy; that is, the same average 
energy per particle in the two subsystems. 

Now we want to find the maximum of the entropy with respect to Na to confirm 
that it occurs at the equilibrium values of Na and Nb = N — Na ■ We find the 
maximum in the usual way, by setting the partial derivative with respect to Na equal 
to zero. 

—^S{Ea, Na] Eb^Va, ^ ^ [Sa{Ea. Pa, Na)+Sb{Eb,Vb,N - N^)] 

= 0 (7.8) 

This gives us another equilibrium condition, in analogy to eq. (7.5), which will also be 
important in thermodynamics. 

OSa OSb 


ONa BNb 


(7.9) 
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The partial derivative of the entropy with respect to N is rather complicated. 
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(7.10) 


The condition for equilibrium is to set the partial derivatives with respect to the 
number of particles equal in the two subvolumes. 
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(7.11) 


Because of the equilibrium with respect to energy, we also have eq. (7.7). 


Ea _ Eb 


(7.12) 


Combining eqs. (7.11) and eqs. (7.12), we find the equation for equilibrium with 
respect to Na- 


Na _ Nb 

14 “ 14 


(7.13) 


As expected from eq. (4.9), the number of particles per unit volume is the same in 
both subsystems. 


7.3 The Volume-Dependence of the Entropy 

An interesting and important feature of the expression we have derived for the entropy 
of the classical ideal gas in eq. (7.2) is that it also correctly predicts the results for a 
kind of experiment that we have not yet discussed. 

Consider a cylinder, closed at both ends, containing an ideal gas. There is a freely 
moving partition, called a piston, that separates the gas into two subsystems. The 
piston is made of a diathermal material, so it can transfer energy between the two 
systems, but it is impervious to the particles. There are Na particles on one side of the 
piston and Nb particles on the other. Since the piston can move freely, the volumes 
of the subsystems can change, although the total volume, V = Va + Vb, is fixed. 
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Because the piston can conduct heat, the average energy per particle will be the 
same on both sides of the piston, as derived in Chapter 6. 


{Ea) ^ 

Na Nb 


(7.14) 


The maximum of the entropy with respect to the position of the piston is then 
found by setting the derivative of the total entropy with respect to Va equal to zero. 

— S{EA,VA,NA\EBy - Va,Nb) = [SA{EAyA,NA) + SBiEB,V-VA,NB)] 
OVa OVa 

= -^^SA{EAyA,NA) - -^;^SB{EByB,NB) 
oVa oVb 


= 0 


(7.15) 


This gives us the equilibrium condition in a familiar form. 

OSa _ OSb 
OVa OVb 


(7.16) 


Since the partial derivative of the entropy with respect to volume is 
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(7.17) 


and the equilibrium condition that the particle density is the same in both subsystems 
is correctly predicted. 


Vb 


(7.18) 


How did this happen? It is nice to know that it is true, but why should the 
maximum of the entropy with respect to the position of the piston produce the 
correct equilibrium volumes? The answer can be found by calculating the probability 
distribution for the position of the piston. 

Under the usual assumption that all configurations are equally likely (subject to 
the constraints), the joint probability density for the positions of the particles and the 
position of the piston should be a constant, which we denote as Y. 


P (U4, {EaAi = 1,..., Na}, {tbAj = 1, ■ • ■, Nb}) = Y 


(7.19) 














76 C/ass/ca/ Gases; Ideal and Otherwise 


To find the marginal probability distribution P(Va), we simply integrate over the 
positions of every particle, noting that Na particles are restricted to volume Va, and 
JVb particles are restricted to volume Vb- 


P 


(Va) = f P{VA,{rA,i\'>-=^^---y^A},{rB,j\j = 

JVA .VFt 

(7.20) 




To find the maximum of P {Va) under the condition that Vb = V — Va, we 
differentiate with respect to Va, and set it equal to zero. 

j^P (Va) = YNaV^^-W^^ - YNbV^^V - Va)^‘^-^ = 0 (7.21) 

OVA 

Solving eq. (7.21), we find the hoped-for result. 


Va Vb 


(7.22) 


The key point is that the logarithm of the volume dependence of P (Va) is exactly 
the same as that of the entropy. In every case, defining the entropy as the logarithm 
of the probability (to within additive and multiplicative constants) gives the correct 
answer. 


7.4 Indistinguishable Particles 

In Chapter 4 we calculated the configurational entropy of a classical ideal gas of 
distinguishable particles. On the other hand, from quantum mechanics we know that 
atoms and molecules of the same type are indistinguishable. It is therefore interesting 
to ask in what way the probability distribution, and consequently the entropy and 
other properties of the composite system, would change if we were to assume that the 
particles were indistinguishable. 

For the following derivation, recall that particles are regarded as distinguishable if 
and only if the exchange of two particles produces a different microscopic state. If the 
exchange of two particles does not produce a distinct microscopic state, the particles 
are indistinguishable. 

The question is how to modify the derivation of eq. (4.4) in Chapter 4 to account 
for indistinguishability. 

The first consequence of indistinguishability is that the binomial coefficient intro¬ 
duced to account for the permutations of the particles between the two subsystems 
must be eliminated. Since the particles are indistinguishable, the number of micro¬ 
scopic states generated by exchanging particles is equal to 1. 

The second consequence is that the factors {Va/V)^^{1 — Va/V)^~^^ in eq. (4.4) 
must also be modified. 
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Consider our basic assumption that the probability density for the positions of 
the particles must be a constant, with the value of the constant determined by the 
condition that the probability density be normalized to 1. 

For N distinguishable particles in a three-dimensional volume V, the normalization 
constant is clearly V~^. The probability of finding a specific set of Na particles in 
subvolume Va and the rest in subvolume Vb is therefore . 

However, calculation for indistinguishable particles is somewhat more difficult. 
We cannot label individual particles in a way that distinguishes between them. We 
can, however, label the particles on the basis of their positions. If two particles are 
exchanged, their labels would also be exchanged, and the state would be unchanged. A 
simple way of achieving this is to introduce a three-dimensional Cartesian coordinate 
system and label particles in order of their x-coordinates; that is, for any microscopic 
state, particles are labeled so that Xj < Xj+i. For simplicity, we will assume that the 
subsystems are rectangular, with edges parallel to the coordinate axes. The lengths 
of the subsystems in the x-direction are La and Lb, and the corresponding areas are 
Aa and Ab- Naturally, the volumes are given by Va = AaLa and Vb = AbLb- 

We will follow the same procedure established in Chapters 4 and 6 and assume the 
probability distribution in coordinate space to be a constant, which we will denote by 
Y{N,V). 


P{{r,})=Y{N,V) 


(7.23) 


The probability of finding Na particles in subvolume Va and Nb = N — Na in 
subvolume Vb is found by integrating over coordinate space. The integrals in the y- 
and z-directions just give factors of the cross-sectional areas of the two subsystems. 
The integrals over the x-coordinates must be carried out consistently with the labeling 
condition that Xj < x^+i. 

pLa /*^2 

P{Na,Nb) = Y{N,V)A^^A^^ dxN^ dxN^-f- dxi 
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do 


dx\ 


(7.24) 


I have used primes to indicate the integrals over the x-coordinates in subsystem B. 
The integrals in eq. (7.24) are easily carried out by iteration. 


I V^^ 

P{Na,Nb)=Y{N,V)^ ^ 


yNB 


NaI / \ Nb\ 


(7.25) 


This gives the probability distribution of identical particles, except for the determina¬ 
tion of the normalization constant, Y{N,V). 

It is clear that the dependence of P{Na, Nb) on its arguments for indistinguishable 
particles in eq. (7.25) is exactly the same as that for distinguishable particles in 
eq. (4.4). This means that we can determine the value of the normalization constant 
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by comparison with the normalized binomial distribution in eq. (3.47). 


nN.v) = ^ 


(7.26) 


The full probability distribution for identical particles is found by substituting 
eq. (7.26) in eq. (7.25). 



(7.27) 


This expression is identical to the probability distribution for distinguishable particles 
in eq. (4.4). 

Since the entropy is given by the logarithm of the probability distribution, the 
entropy of a classical gas is exactly the same for distinguishable and indistinguishable 
particles. 

The result that the entropy is exactly the same for classical systems with distin¬ 
guishable and indistinguishable particles might be surprising to some readers, since 
most of the literature for the past century has claimed that they were different. 
My claim that the two models have the same entropy depends crucially on the 
definition of entropy that I have used. However, I think we might take as a general 
principle that two models with identical properties should have the same entropy. 
Since the probability distributions are the same, their properties are the same. It 
is hard to see how any definition that results in different entropies for two models 
with identical properties can be defended. 


7.5 Entropy of a Composite System of Interacting Particles 

Up to this point we have considered only an ideal gas, in which there are no interactions 
between particles. In real gases, particles do interact: particles can attract each other 
to form liquids and they can repel each other to make the liquid difficult to compress. 
Fig. 7.1 shows a typical interatomic potential, which is significantly different from zero 
only over very short distances. 

Even though interatomic potentials are very short-ranged, they can have important 
effects, including the formation of liquid and solid phases, with phase transitions 
between them. We will return to this topic in Chapters 19 and 30. 

To generalize our analysis of the entropy of the classical ideal gas to include 
interacting particles, it is useful to go back and re-derive the entropy of the ideal 
gas, without taking advantage of the independence of the positions and the momenta. 
Our basic assumption is still that of a uniform probability in phase space, consistent 
with the constraints on the composite system. 




Entropy of 3 Composite System of Interscting Psrticles 79 


(p(r) 



r 


a few nanometers 


Fig. 7.1 Schematic plot of typical interatomic potential. The potential is significantly different 
from zero only over a few nanometers. 

7.5.1 The Ideal Gas Revisited 

We begin with our assumption from Section 1.4 that the probability distribution in 
phase space is uniform, subject to the constraints. The physical constraints are: 

1. Na particles are in subsystem A and TVs in subsystem B. 

2. Subsystem A has an energy Ea and subsystem B has an energy Eb. 

Instead of doing the integrals separately for the configurations and the momenta, 
we can write the final result for P{Ea,Va,Na', Eb,Vb, Nb) as an integral over all 
of phase space, with Dirac delta functions imposing the energy constraints as before. 
This does not simplify matters for the ideal gas, but it will make the generalization 
to real gases easier. 

Calculating the probability distribution for the energies, volumes, and numbers 
of particles of each subsystem is straightforward, but the expression becomes a little 
unwieldy. While it is great fun for a teacher to write it across two or three blackboards 
so that students can see it as a whole, the constraints of a book make it somewhat more 
difficult to write out completely. To make it manageable, we will use the same compact 
notation that we used earlier for the integrals over phase space. The SiV^i-dimensional 
integral over the positions of the particles in subsystem A will be indicated as JdqA, 
and similarly for the other variables. Integrals over quantities without subscripts refer 
to the entire composite system. 

Write the Hamiltonian of the subsystem as 



2m 


(7.28) 
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where a = A or B to indicate the subsystems. If the subscript is omitted, then the 
expression refers to the entire composite system. To make the notation more compact 
we will write to indicate Hiy{Ea, 14 ,, N^). 

The marginal probability distribution for the energies, volumes, and number of 
particles of each subsystem is then found by integrating over all of the space. 


P{Ea^Va, Na', Eb,Vb, Nb) = 

TV! / dqA f dpA f dqB f dpBS{EA - Ha)6{Eb - Hb) , ^ . 

Na\Nb\ JdqJdp6{E-H) ’ 

Because the particles in the two subsystems do not interact with each other, 
eq. (7.29) can be written in a more compact form by introducing a function 

na{Ea,Vc,Na) = | jjdpaS{Ea - Hdqc, Pa)) (7.30) 

where we have written Ha{qayPa) for clarity. Note that we have included a factor of 
h-^Na jjj expression for ida- This factor is not required classically, but it is allowed 
since we can always multiply the right side of eq. (7.29) by 1 = . We 

will see later that this simple modification produces consistency between the classical 
result for the ideal gas and the classical limit of a quantum ideal gas. 

With this notation, eq. (7.29) becomes 


P{Ea,Va,Na-,Eb,Vb,Nb) 


nA{EA,VA.NA)nB{EB,VB.NB) 

n{E,V,N) 


(7.31) 


Taking the logarithm of the probability distribution in eq. (7.31), in analogy to 
eq. (4.18), we find the expression for the entropy of the composite system. 


StotiEA, Va, Na; Eb,Vb,Nb) = kin [P{Ea, 14, Na; Eb,Vb,Nb)] + S{E, 17, TV) 

= Sa{Ea,Va,Na) + Sb{Eb,Vb,Nb) (7.32) 


The entropy of an isolated system is given by the logarithm of fl. 


S{E, V,N) = k In n{E, 17, TV) (7.33) 

Eq. (7.32) shows that the total entropy of an ideal gas is additive. It will be left 
as an exercise to show that this expression for the entropy of a classical ideal gas is 
identical to that in eq. (7.2)—including the value of the constant X given in eq. (7.3). 

Note that we have introduced a new constant, Ti, into eq. (7.31). It is easy to confirm 
that this expression for the probability distribution is still equivalent to eq. (7.29) 
for any value of h, so that this new constant is completely arbitrary, and therefore 
meaningless within classical statistical mechanics. It has been introduced solely for the 
purpose of ensuring consistency with quantum statistical mechanics. It is a remarkable 
fact that the choice of h as Planck’s constant produces agreement with quantum 
statistical mechanics in the classical limit. This probably seems rather mysterious at 
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this point, but it will become clear when we return to this question in Chapters 26, 
27, and 28. 

7.5.2 Generalizing the Entropy to Interacting Particles 

We do not need to make any changes in our fundamental assumptions about the 
probability distribution in phase space to find the entropy of a system with interacting 
particles. We still assume a uniform probability distribution in phase space, subject 
to the physical constraints of our composite system. The only difference is that the 
energy now includes interaction terms. However, because it is possible for a molecule in 
one subsystem to interact with a molecule in the other subsystem, the separability of 
the entropy of the composite system into the sum of the entropies for each subsystem 
is no longer trivial. 

First write the Hamiltonian of the composite system. 


3N 1^12 3N 3N 



(7.34) 


j = l i=l,i^j 


or 



3Na 3Na 


3Nb 3Nb 


+ E E rj,A) + 


j=l 


j = l 


3Na 3Nb 



(7.35) 


j=l i=l 


This equation is written two ways. First, as a single, composite system, and then 
broken into pieces, representing the subsystems. The last term on the right in eq.(7.35) 
represents the interactions between particles in different subsystems. This is the term 
that causes difficulties in separating the entropy of the composite system into a sum 
of entropies of the subsystems. 

We can define the Hamiltonian of subsystem A. 



(7.36) 


i = l 


The limit oil > j in the second sum over positions is there to prevent double counting. 
Similarly, we can define the Hamiltonian of subsystem B. 



j=l i=l,i^j 


2m 


(7.37) 
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However, because of the interaction term linking the two subsystems, 


3Na 3Nb 

HAsiqA, qs) = EE <P(ri,A,rj^B) (7.38) 

j=i i=i 

the total Hamiltonian is not just the sum of Ha and Hg. 

H{q,p) = HA{qA,PA) + HgiqB^PB) + HAB{qA,qB) (7.39) 

To analyze a composite system of interacting particles, it turns out to be easier to 
use delta functions to specify Ea and E in calculating the marginal probability than 
it is to specify Ea and Eg- 

P{Ea,Va,Na;E-Ea,Vb,Nb) = 

Nl f dqA f dpA f dqg f dpBS(EA - Ha)S{E - H) 

Na\Nb\ JdqJdpS{E-H) ^ ’ 

Now modify eq. (7.40) by replacing E with Ea + Eg and using the presence of the 
first delta function 6{Ea — Ha) to replace Ea with Ha in the second delta function. 


P{Ea,Va,Na-,Eb,Vb,Nb) = 

N\ JdqA JdpA Jdqg JdpgSjEA - Ha)6{Eb - Hg- Hab) n a\\ 

NaINbI JdqJdpS{E - H) ^ ’ 

The interaction term, Hab, prevents eq. (7.41) from separating exactly into factors. 

Fortunately, the range of the interactions between molecules in most materials 
(or between particles in colloids) is short in comparison with the size of the system. 
Therefore, the contributions to Hab are only significant for molecules in different 
subsystems that are close to each other. Since the range of molecular interactions is 
only a few nanometers, this means that only pairs of molecules very close to the inter¬ 
face between the two subsystems contribute at all. The energy corresponding to these 
direct interactions between molecules in different systems is therefore proportional to 
the size of the interface, which scales as the surface area, or ■ 

In terms of the number of particles, the contribution of the direct interactions 
between molecules in different subsystems is proportional to Nd , so that the relative 
contribution is proportional to jNa = Ha ■ For example, if Na ~ 10^^, then 

Na ~ 10“^. This is can be regarded as a very good approximation. However, it 
should be noted that the effect of these interface interactions is considerably larger 
than the statistical fluctuations, which scale as Xj^jHa- Interface or surface effects 
must also be taken more seriously for colloids, which might only contain 10^^ particles, 
so the relative contribution of the interface interactions could be of the order 10“^. 
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If the interaction term, Hab, can be neglected, eq. (7.41) can be separated into 
factors. 


P{Ea,Va,Na;Eb,Vb,Nb) = 

N\ JdqA JdpA6{EA - Ha) JdqB JdpBS{EB - Hb) . . 

Na\Nb\ JdqJdp6{E-H) ^ ’ ’ 

The factors in eq. (7.42) can be made explicit by introducing a generalized function, 
^a{Ec,,Va,Na) = | jJdpa5{Ea - H^) (7.43) 


and writing 


P{Ea,Va,Na-,Eb,Vb,Nb) 


nA{EA,VA,NA)nB{EB,VB,NB) 

n{E,V,N) 


(7.44) 


As before, the subscript a can take on the values of A or B to represent a subsystem, 
or be missing entirely to represent the full system. The parameter h is still completely 
arbitrary within classical statistical mechanics. As mentioned above, h will be iden¬ 
tified as Planck’s constant in Chapters 26, 27, and 28 to ensure consistency with the 
classical limit of the corresponding quantum system. 

Now that we have expressed the probability distribution for interacting classical 
particles as a product of terms, we can define the entropy by taking the logarithm and 
multiplying it by the constant. The result is formally the same as eq. (7.32) for ideal 
gases. 


Stot{EA. Va, Na; Eb,Vb,Nb) = kin [P{Ea, Va, Na', Eb,Vb,Nb)] + S{E, P, N) 

= SAiEA,VA,NA) + SBiEB,VB,NB) (7.45) 


The entropy terms of individual systems are again given by logarithm of the 
functions. 


Sa{Ea,Va,Na) = fc In fla (A^ , P„, A^^) 


= fc In 


1 


dpaS{Ea 



(7.46) 


7.6 The Second Law of Thermodynamics 

As was the case for ideal gases, the probability distributions for energy, volume, and 
number of particles in composite systems of interacting particles are very narrow— 
generally given by the square root of the number of particles. The width of the 
probability distributions will therefore again be much smaller than experimental errors. 
For systems of interacting particles, the observed values of the energy, volume, and 
number of particles will agree with the location of the maxima of the corresponding 
probability distributions to within the accuracy of the experiment. 
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A comparison of eq. (7.42) for the probabilities in a compound system of interacting 
particles and eq. (7.44) for the entropy of the composite system shows that the location 
of the maximum of the entropy gives the equilibrium values of the quantity of interest 
when the corresponding internal constraints are released. 

This is an extremely important result for two reasons. 

First, it allows us to find the equilibrium values of the energy, volume, or number 
of particles for any experimental situation, once the entropy is known. 

Second, if the entropy is maximized for unconstrained equilibrium, it necessarily 
follows that any constrained macroscopic state of the composite system must have a 
lower total entropy. This is the Second Law of Thermodynamics. 

If the composite system is in any constrained macroscopic state—that is, if there 
are any constraints on the energy, volume, or number of particles in any subsystem— 
the total entropy of the composite system cannot decrease if those constraints are 
removed. 

7.7 Equilibrium between Subsystems 

For a composite system composed of two general, classical subsystems, the 
total entropy is the sum of the entropies of the subsystems, as indicated in 
eq. (7.45). 


Stot{EA,VA,NA-,EB,VB,NB) = Sa{Ea,Va,Na) + Sb{Eb,Vb,Nb) (7.47) 


Since the total entropy is maximized upon release of a constraint, we can determine 
the new equilibrium parameters by setting the appropriate partial derivative of the 
total entropy equal to zero. 

For example, consider a situation in which the volumes and numbers of particles 
are held fixed in the subsystems, but the subsystems are connected by a diathermal 
wall. Equilibrium is found from the partial derviative with respect to Ea, with 
Eb = E — Ea- 



— Sa{Ea. Va, Na) + ^Sb{E - Ea, Vb,Nb) = 0 
oEa oEa 


(7.48) 


Since Eb = E — Ea, it must be that BEa/OEb = —1, and we can rewrite eq. (7.48) 
in a symmetric form. 




(7.49) 


or 


dSA OSb 


(7.50) 


OEa BEb 
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Eq. (7.50) is extremely important. Beyond giving us an algorithm for finding the 
equilibrium value of , it tells us that the condition of equilibrium is that a derivative 
of the entropy with respect to the energy has the same value for two subsystems in 
thermal contact. 

Similar arguments show that the condition for equilibrium with respect to volume 
when two subsystems are separated by a moveable piston is given by 


dSA dSB 
dVA OVb 


(7.51) 


and equilibrium with respect to particle number when a hole is made in the wall 
between two subsystems is given by 


dSA dSB 
ONa ONb 


(7.52) 


These last three equations are extremely important, both as a means of solving 
problems and as the basis for establishing the connection between statistical mechanics 
and thermodynamics. The explicit relationships between these derivatives and familiar 
quantities like temperature and pressure will be the subject of the following chapter. 
However, the first and arguably the most important consequence of these equations is 
presented in the next section. 


7.8 The Zeroth Law of Thermodynamics 

An immediate consequence of eqs. (7.50), (7.51), and (7.52) is that if two systems are 
each in equilibrium with a third system, they must also be in equilibrium with each 
other. 

Assume systems A and B are in equilibrium with respect to energy, then eq. (7.50) 
holds. If systems B and C are also in equilibrium with respect to energy, then we 
have a similar equation for the partial derivatives oi Sb and Sc with respect to 
energy. 

SEa OEb dEc ^ ’ 

Since the partial derivatives with respect to energy must be the same for systems A 
and C, they must be in equilibrium with respect to energy. 

Clearly, this argument is equally valid for the volume or the number of particles. 
We therefore have a general principle that if two systems are each in equilibrium with 
a third system, then they are in equilibrium with each other. 

This is the Zeroth Law of Thermodynamics. 

The numbering of the laws of thermodynamics might seem rather strange— 
especially the ‘zeroth’. The need to state it explicitly as a law of thermodynamics 
was not recognized during the nineteenth century when thermodynamics was being 
developed, perhaps because it is so fundamental that it seemed obvious. When it 
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was declared a law of thermodynamics, the numbers for the other laws had long 
been well established. Ralph H. Fowler (British physicist and astronomer, 1889- 
1944) is credited with calling it the Zeroth Law to place it in the leading position. 
At this point in the book we have encountered only the Second and Zeroth Laws 
of Thermodynamics. The First and Third Laws are yet to come. 


7.9 Problems 


Problem 7.1 

General forms for the entropy 

We have derived the entropy of a classical ideal gas, which satisfies all the postulates 
of thermodynamics (except the Nernst postulate, which applies to quantum systems). 
However, when interactions are included, the entropy could be a very different function 
of U, V, and N. Here are some functions that are candidates for being the entropy of 
some system. [In each case, A is a positive constant.] 

1. Which of the following equations for the entropy satisfy the postulates of ther¬ 
modynamics (except the Nernst postulate)? 


1 . S = A{UVNf/^ 



2. For each of the valid forms of the entropy in the previous part of the problem, 
find the three equations of state. 

Problem 7.2 

The “traditional” expression for the entropy of an ideal gas of distinguish¬ 
able particles 

It is claimed in many textbooks on statistical mechanics that the correct entropy for 
a classical ideal gas of distinguishable particles differs from what we derived in class. 
The traditional expression is 
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Assume that you have two systems, A and B, with and Nb particles and Va 
and Vb volumes respectively, which are both described by an entropy of this form. 

Consider an experiment in which the volumes are fixed, but a hole is made in the 
wall separating the two subsystems. 

Calculate the location of the maximum of the total traditional entropy to determine 
its prediction for the equilibrium distribution of the energy and the particles between 
the two systems. 

Naturally, your answer should depend on the relative sizes of the fixed volumes, 
Va and Vb- 


8 

Temperature, Pressure, Chemical 
Potential, and All That 


Pressure makes diamonds. 
General George S. Patton 


In this chapter we will discuss the temperature, T, the pressure, P, and the chemical 
potential, fi. We will find explicit expressions for these three quantities in terms 
of partial derivatives of the entropy with respect to energy, volume, and number 
of particles. In doing so, we will complete the foundations of classical statistical 
mechanics. 

In Chapter 9, which begins Part II, we will see how the structure of statistical 
mechanics provides a foundation for thermodynamics. The rest of the chapters in 
Part II complete the development of the theory of thermodynamics. 

In Part III we shall return to classical statistical mechanics. While the equations 
developed in Part I are (in my opinion) the clearest way to understand the assumptions 
of statistical mechanics, the new methods developed in Part III are much more 
powerful for practical calculations. 


8.1 Thermal Equilibrium 

We have seen in Section 7.8 that if two objects are in thermal equilibrium with each 
other (equilibrium with respect to energy exchange), then the partial derivative of 
the entropy with respect to energy must have the same value in both systems. We 
also know that if two systems are in thermal equilibrium, they must be at the same 
temperature. Therefore, the partial derivative of the entropy with respect to energy 
must be a unique function of temperature. One purpose of this chapter is to determine 
the nature of that function. 

After we have determined the relationship between the temperature and the partial 
derivative of the entropy with respect to energy, we will find similar relationships 
between other partial derivatives of the entropy and the pressure and chemical 
potential. 

In all cases, the relationships linking T, P, and /r with partial derivatives of the 
entropy will be valid for all systems. This is a very powerful statement. It means that 
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if we can calculate the entropy as a function of E, V, and N, we can calculate all 
thermal properties of the system. For this reason, the entropy as a function of energy, 
volume, and number of particles. 


S = S{E,V,N) (8.1) 

is known as a ‘fundamental relation’. We will see later that there are a number of other 
functions that also contain the same, complete thermodynamic information about a 
system, which makes them equivalent to eq. (8.1). Such functions are also known as 
fundamental relations. 

8.2 What do we Mean by ‘Temperature’? 

For most of us, temperature is the reading we obtain from a thermometer . The basic 
property of a thermometer is that it undergoes some sort of physical change that can 
be measured when it is heated or cooled. The most readily available thermometer is 
our own body. It shivers when it is cold, and sweats when it is hot. These are subjective 
measures, but they do form the basis for our intuitive understanding of temperature. 

To make the definition of temperature objective, we need to choose something that 
will provide a theoretical and experimental standard, as well as a numerical scale. We 
will use the ideal gas for this purpose. Since we can calculate all of the properties 
of an ideal gas, we can determine how its volume or pressure will change when it is 
heated or cooled. Since we can do experiments on real dilute gases, we can relate our 
definition of temperature to the real world. 

The basic equation we will use to uniquely define the thermodynamic temperature, 
is the ideal gas law. 


PV = NksT (8.2) 

The pressure P in this equation is defined as the average force per unit area. The 
constant ks is called Boltzmann’s constant. It has units of Joules per degree, and 
relates the temperature scale to the energy scale. Eq. (8.2) is often written as 

PV = nRT (8.3) 

where n = N/Na is the number of moles {Na = 6.0221415 x 10^^ being Avogadro’s 
number), and R= NAks is known as the ideal gas constant. The experimental 
identification of the temperature as being proportional to the pressure for fixed volume, 
or the volume for fixed pressure, coincides with our notion that gas expands, or its 
pressure increases, when heated. 

Although the ideal gas law is well known experimentally, to make contact with 
the entropy we must derive it from the properties of the ideal gas found in previous 
sections. What we will actually do in the next section is prove that the ratio PV/N 
is equal to a certain property of a thermal reservoir, and then use eq. (8.2) to 
associate that property with the temperature. This derivation will give us the universal 
relationship between the temperature and dS/dE. 
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8.3 Derivation of the Ideal Gas Law 

The first step in the derivation of the ideal gas law is to derive the Maxwell-Boltzmann 
equation for the probability density of the momentum of one ideal gas particle. We 
will then use this probability density to determine the pressure due to collisions on the 
walls containing an ideal gas. This will lead us to the ideal gas law and the definition 
of the temperature in terms of the partial derivative of the entropy with respect to 
the energy. 

8.3.1 The Maxwell Boltzmann Equation 

We are interested in the properties of a single particle in an ideal gas, and it does not 
matter which particle we choose, since they all have the same probability density. We 
can find that probability density as a marginal density of the full probability density 
in phase space. If the total energy of the system is E, the full probability density is 
given by 


where 

E, N) = 5{E - H{p, q)) (8.5) 

and we have used the compact notation p = {pj\j = 1,2,... ,N} and q = {rj\j = 1, 
2,...,iV}. 

As a first step, we will find the marginal probability density for both the momentum 
and position of particle 1 by integrating P{p,q) in eq. (8.4) over all variables except 
Pi and fq. 

PiPi^n) = j (fp 2 ■ ■ ■ d^PN j d^r 2 ■ ■ ■ d^rtqP(p, q) 

^ niE^V^N) h^. / jd)) ( 8 . 6 ) 

In eq. (8.6), H]s[ is the Hamiltonian of the Wparticle system. 

We can write the integral in eq. (8.6) in terms of the functon H for a system with 
fV — 1 particles. 


n{E- |pi| V2m, y, A - 1) = Jdp 2 ... dp3N 

X jd?‘r2 ■ ■ ■ d^r^S (e— |pip/2m — '^ p'\l2m | (8.7) 






Derivation of the Ideal Gas Law 91 


Note that the energy term, |pip/277i, has been separated from the rest of the 
Hamiltonian because it has not been integrated out. With this notation, eq. (8.6) 
can be written compactly. 


PiPi^n) 


n{E-\pi\^/2m,V,N-1) 
Nh^n{E, V, N) 


( 8 . 8 ) 


Since P{pi,fi) does not depend explicitly on fi, we can easily integrate it out to 
obtain 


p.. . ^ H n{E-\p,\y2m,V,N-l) 
Nh^ fl{E,V,N) 


(8.9) 


We would like to take advantage of the fact that |pip/2m is very small in 
comparison to E. We can exploit this by taking the logarithm of eq. (8.9), treating 
|pip/2m as a small perturbation and keeping only the leading terms. 

InP(pi) = lnn(iJ — IpiP/2771, N — 1) 

-lnH(P,t/,iV) + ln(^) 

« lnf}(P,y,iV- 1) - ^^^lnn{E,V,N-l) 

-lnn{E,V,N) + ln(^^'^ ( 8 . 10 ) 


The higher-order terms in the expansion are extremely small because the average value 
of |pip/2m is a factor of N smaller than E. 

The first thing to note about eq. (8.10) is that only the second term depends on pi. 
The second thing to note about eq. (8.10) is that the approximation 

±lnn{E,V,N-l)^ ^lnniE,V,N) ( 8 . 11 ) 

is extremely good. The error is only of order 1/N. 

The derivative of In 12 with respect to energy turns out to be so important that it 
has been assigned a special Greek letter. 

/3= 4;lnl2(£;,H,iV) (8.12) 


We can now rewrite eq. (8.10) in a much simpler form. 

P{pi) = —/3|pip/2m-I-constants 


(8.13) 
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Since the constants in this equation are determined by the normalization condition, 
we can complete the derivation of P{pi). 

= <«■«) 

This is the Maxwell-Boltzmann probability density for the momentum of a single 
particle. 

The expansion used in deriving P{pi) is extremely valuable in statistical mechanics. 
This is the first time it appears in this book, but far from the last. 


Note that the three components of the momentum are independent, so that the 
probability density of a single component can be found easily by integrating out the 
other two. 

P(p..J = \/^exp(-;3|^) (8,15) 

8.3.2 The Pressure in an Ideal Gas 

From the Maxwell-Boltzmann probability density for the momenta, we can calculate 
the pressure by integrating over the collisions that occur during a time At. 

Consider a flat portion of the wall of the container of an ideal gas with area A. (The 
assumption of flatness is not necessary, but it makes the derivation much easier.) Define 
a coordinate system such that the x-axis is perpendicular to the wall. To simplify the 
problem, note that the y- and ^-components of a particles momentum do not change 
during a collision with the wall, so that they do not transfer any momentum to the 
wall and do not affect the pressure. This reduces the calculation to a one-dimensional 
problem. 

By Newton’s Second Law, the average force on the wall during a time-period At 
is given by the total momentum transferred to the wall during that period. 

FAt= ( Ap^P{p^,At)dp^ (8.16) 

J collisions 

P{px, At) is the probability of a particle hitting the wall during the time At, and the 
integral goes over all particles that hit the wall during that period of time. Since we 
intend to make At small, we do not have to include particles that might hit the wall 
a second time after bouncing off a different part of the container. 

Let the wall be located at a; = 0, with the particles confined to a; < 0. Then particles 
will hit the wall only if they are within a distance Atvx = Atpx/m of the wall, where 
m is the mass of a particle. Letting the area of the wall be denoted by A, the volume 
of particles that will hit the wall is AAtpx/ni. The average number of particles in 
that volume is NAAtpx/Vm, where N is the total number of particles, and V is the 
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total volume. If we first multiply the number of particles in the volume by P(px) 
(the probability density of the momentum from eq. (8.14)), then multiply that by the 
momentum transfer, = 2px, and finally integrate over the momentum, we have 
the total momentum transfer to the wall during At, which is equal to the average force 
times At. 

Note that the integral extends only over positive momenta; particles with negative 
momenta are moving away from the wall. 

Using the definition of the pressure as the force per unit area, P = F/A, eq. (8.17) 
becomes an equation for the pressure. 


2N 

Vm 



pldpx 


(8.18) 


The integral in eq. (8.18) can be carried out by the methods discussed in 
Section 3.10. You could also look up the integral in a table, but you should be able to 
work it out by yourself. The result is surprisingly simple. 


PV = A/3-1 


(8.19) 


Eq. (8.19) should be compared with the usual formulation of the ideal gas law. 


PV = NkeT 


( 8 . 20 ) 


where is known as Boltzmann’s constant. This comparison allows us to identify /3 
with the inverse temperature. 

( 8 . 21 ) 

Because of the Zeroth Law, we can use an ideal-gas thermometer to measure the 
temperature of anything, so that this expression for (3 must be universally true. 


8.4 Temperature Scales 

Although we are all familiar with the thermometers that we use every day, the 
temperature scale in the previous section is rather different than those commonly 
found in our homes. First of all, eq. (8.2) makes it clear that T cannot be negative; 
but both temperature scales in common use—Celsius and Fahrenheit—do include 
negative temperatures. 

The Celsius temperature scale is defined by setting the freezing point of water equal 
to 0° C, and the boiling point of water equal to 100° C. The Celsius temperature is then 
taken to be a linear function of the temperature T given in eq. (8.2). Operationally, 
this could be roughly carried out by trapping a blob of mercury in a glass tube at 
high temperature, and measuring the position of the mercury in freezing water and 
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in boiling water. I actually carried out this experiment in high school, before people 
were quite aware of the toxicity of mercury. 

Using the Celsius temperature scale we can extrapolate to zero volume of the gas. 
Done carefully, it extrapolates to a temperature of —273.15° C, which is known as 
‘absolute zero’ because it is the lowest possible temperature. 

For the rest of this book we will use the Kelvin temperature scale, which is shifted 
from the Celsius scale to agree with eq. (8.2) and make zero temperature fall at 
absolute zero. 


T(°K) =T(°C) + 273.15 


( 8 . 22 ) 


Although eq. (8.22) seems quite simple, it is easy to forget when doing problems. 
A very common error in examinations is to use the Celsius temperature scale when 
the Kelvin scale is required. 


Since we are following tradition in defining the Celsius and Kelvin temperature scales, 
the value of Boltzmann’s constant is fixed at ks = 1.380658 x If we 

were to have thermometers that measure temperature in Joules, Boltzmann’s constant 
would simply be 1. 


8.5 The Pressure and the Entropy 

From the explicit expression for the entropy of the classical ideal gas in eq. (7.2), we 
can find the derivative of the entropy with respect to the volume. 




E,N 








(8.23) 


This relationship is only true for the ideal gas. By comparing eq. (8.23) with the ideal 
gas law, eq. (8.2), we find a simple expression for the partial derivative of entropy with 
respect to volume that is true for all macroscopic systems. 


5S'\ _ kP 


(8.24) 


At this point we will make the standard choice to set the constant k that we first 
introduced in eq. (4.16) equal to Boltzmann’s constant, k = ks- Eq. (8.24) now 
becomes 
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T 


(8.25) 


This is the formal thermodynamic relationship between entropy and pressure. 
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8.6 The Temperature and the Entropy 

By comparing eqs. (7.30), (8.12), and (8.21), we can see that 

Because of the Zeroth Law, this relationship must be valid for all thermodynamic 
systems. It is sometimes used as the definition of temperature in books on thermody¬ 
namics, although I have always felt that eq. (8.26) is too abstract to be a reasonable 
starting point for the study of thermal systems. One of the purposes of the first part 
of this book is to explain the meaning of eq. (8.26). If that meaning is clear to you 
at this point, you will be in a good position to begin the study of thermodynamics in 
the next part of the book. 

Since we know the entropy of the classical ideal gas from eq. (7.2), we can evaluate 
the derivative in eq. (8.26) explicitly. 
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(8.27) 


The last equality gives us the energy of the classical ideal gas as a function of 
temperature. 


E = ^NkeT 
2 


(8.28) 


An interesting consequence of eq. (8.28) is that the average energy per particle is 
just 


f = IksT (8.29) 

which is independent of the particle mass. This is a simple example of the equipartition 
theorem, which we will return to in Chapter 19. 

8.7 The Entropy and the Chemical Potential 

The chemical potential, /r, is related to the number of particles in much the same 
way that the temperature is related to the energy and the pressure is related to the 
volume. However, while we do have thermometers and pressure gauges, we do not have 
a convenient way of directly measuring the chemical potential. This makes it difficult 
to develop our intuition for the meaning of chemical potential in the way we can for 
temperature and pressure. 
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On the other hand, it is straightforward to define the chemical potential in analogy 
with eqs. (8.26) and (8.25). 


( = zR. 

T 


(8.30) 


For the classical ideal gas, we can find an explicit equation for the chemical 
potential from the equation for the entropy. 
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(8.31) 


The complexity of this expression probably adds to the difficulty of acquiring an 
intuitive feeling for the chemical potential. Unfortunately, that is as much as we can 
do at this point. We will revisit the question of what the chemical potential means 
several times in this book, especially when we discuss Fermi-Dirac and Bose-Einstein 
statistics in Chapters 26, 27, and 28. 


8.8 The Fundamental Relation and Equations of State 

We have seen that derivatives of the entropy with respect to energy, volume, and 
number of particles give us three equations for the three variables T, P, and /r, so that 
we have complete information about the behavior of the thermodynamic system. For 
this reason. 


S = S{E,V,N) (8.32) 

is called a fundamental relation. Specifically, it is called the fundamental relation in 
the entropy representation, because the information is given in the functional form of 
the entropy. There are also other representations of the same complete thermodynamic 
information that we will discuss in Chapter 12. 

The derivatives of the entropy give what are called ‘equations of state’, because 
they give information concerning the thermodynamic state of the system. Since there 
are three derivatives, there are three equations of state for a system containing only 
one kind of particle. The equations of state for the classical ideal gas are given in 
eqs. (8.28), (8.2), and (8.31). 

Note that although it is quite common to hear PV = NksT referred to as ‘the’ 
equation of state, it is only valid for the classical ideal gas, and even then it is only 
one of three equations of state. 

For a general thermodynamic system, the three equations of state are independent. 
If all three are known, the fundamental relation can be recovered (up to an integration 
constant). 
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On the other hand, as we will see in Chapter 13, only two of the equations of state 
are independent for extensive systems—that is, systems for which the entropy, energy, 
volume, and number of particles are all proportional to each other. In that case, only 
two equations of state are needed to recover the fundamental relation. 

While the fundamental relation contains complete information about a thermody¬ 
namic system, a single equation of state does not. However, equations of state are still 
very important because, under the right circumstances, one equation of state might 
have exactly the information required to solve a problem. 


8.9 The Differential Form of the Fundamental Relation 


Much of thermodynamics is concerned with understanding how the properties of a 
system change in response to small perturbations. For this reason, as well as others 
that we will see in later chapters, the differential form of the fundamental relation is 
very important. 

A small change in the entropy due to small changes in the energy, volume, and 
number of particles can be written formally in terms of partial derivatives. 


dS = 



dE + 

V,N 



dV + 

E,N 



(8.33) 


From the equations of state we can rewrite eq. (8.33) in a very useful form. 


dS = 



(8.34) 


This is known as the differential form of the fundamental relation in the entropy 
representation. 


8.10 Thermometers and Pressure Gauges 

As mentioned above, the most important characteristic of a thermometer is that it 
shows a measurable and reproducible change when its temperature changes. 

The next most important characteristic of a thermometer is that it should be 
small in comparison to the system in which you are interested. Energy will have to 
flow between the system and the thermometer, but the amount of energy should be 
small enough so that the temperature of the system of interest does not change. 

A ‘pressure gauge’ must satisfy the same condition of ‘smallness’. It must have 
some observable property that changes with pressure in a known way, and it must be 
small enough not to affect the pressure in the system being measured. 

8.11 Reservoirs 

In studying thermodynamics it will often be useful to invoke a thermal reservoir at 
a fixed temperature, so that objects brought into contact with the reservoir will also 
come to equilibrium at the same temperature. The most important characteristic of 
a ‘reservoir’ is that it must be very large in comparison to other systems of interest. 
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SO that its temperature does not change significantly when brought into contact with 
the system of interest. 

We will also find it useful to extend the concept of a reservoir to include a source 
of particles at a fixed chemical potential, or a large container of gas that can maintain 
a fixed pressure when connected to the system of interest through a piston. 


8.12 Problems 


Problem 8.1 

Equilibrium between two systems 

Consider a mysterious system X. The energy-dependence of entropy of system X has 
been determined to be 



where A and B are positive constants. 

Suppose that system X has been put into thermal contact with a monatomic ideal 
gas of N particles and the composite system has come to equilibrium. 

1. Find an equation for Ux in terms of the energy of the ideal gas. 

[You do not have to solve the equation.] 

2. What is the temperature of the ideal gas as a function of Ux, Vx , and iVx? 
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The Postulates and Laws 
of Thermodynamics 


In this house, we OBEY the laws of thermodynamics! 
Homer Simpson (Dan Castellaneta, American actor and writer) 


9.1 Thermal Physics 

With this chapter we begin the formal study of thermodynamics, which represents a 
detour from the development of the foundations of classical statistical mechanics that 
we began in Part I. We will return to statistical mechanics in Parts III and IV. 

The theories of statistical mechanics and thermodynamics deal with the same 
physical phenomena. In many cases it is possible to derive exactly the same equations 
from either theory, although not necessarily with equal ease. In no case do the 
two theories contradict each other. Nevertheless, they do have different origins, and 
are based on very different assumptions. Because of the different points of view 
that accompanied the development of thermodynamics and statistical mechanics, 
it is easy to become confused about what is an assumption and what is a derived 
result. 

The task of keeping assumptions and results straight is made more difficult by the 
traditional description of thermodynamic ideas by the terms ‘postulates’ and ‘laws’, 
since a postulate or law in thermodynamics might also be regarded as being derived 
from statistical mechanics. 

9.1.1 Viewpoint: Statistical Mechanics 

The foundations of statistical mechanics lie in the assumption that atoms and 
molecules exist and obey the laws of either classical or quantum mechanics. Since 
macroscopic measurements on thermal systems do not provide detailed information 
on the positions and momenta (or the many-particle wave function) of the 10^° or more 
atoms in a typical object, we must also use probability theory and make assumptions 
about a probability distribution in a many-dimensional space. 

On the basis of the assumptions of statistical mechanics, we can write down a 
formal expression for the entropy as the logarithm of the probability of a composite 
system, as we did for classical statistical mechanics in Part I. In some simple cases. 
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such as the classical ideal gas, we can even obtain a closed form expression for the 
entropy. In most cases we must make approximations. 

In Part I, we found an explicit expression for the entropy of the classical ideal gas, 
which allows us to understand the properties of the entropy for this simple system. 
We also presented arguments that certain properties of the entropy should not change 
when interactions between particles are included in the calculation. 

The point of view taken in this book is that statistical mechanics is the more 
fundamental theory. Thermodynamics is based on assumptions that can be under¬ 
stood in the context of statistical mechanics. What we will call the ‘Postulates of 
Thermodynamics’ and the ‘Laws of Thermodynamics’ are, if not theorems, at least 
plausible consequences of the theory of statistical mechanics. 

Molecular theory and statistical mechanics provide a fundamental explanation and 
justification of thermodynamics. We might as well take advantage of them. 


9.1.2 Viewpoint: Thermodynamics 

Thermodynamics was defined in Chapter 1 as the study of everything connected with 
heat. The field was developed in the nineteenth century before the existence of atoms 
and molecules was accepted by most scientists. In fact, the thermodynamics of the 
nineteenth century was expressed in terms of the mass of a system, rather than the 
number of particles it contains, which is standard today. 

Because thermodynamics was developed without benefit of the insight provided 
by molecular theory, its history is characterized by ingenious but rather convoluted 
arguments. These arguments led eventually to a formal system of postulates based on 
the concept of entropy, from which all of thermodynamics could be derived. 

From the point of view of thermodynamics, everything starts with the postulates 
presented later in this chapter. Even though these postulates can be traced back to 
statistical mechanics, they provide a consistent starting place for a self-consistent 
theory that produces complete agreement with experiments in the real world. 

Thermodynamics can also be regarded as complementary to statistical mechan¬ 
ics. It gives different insights into physical properties. In many cases, thermody¬ 
namics is actually more efficient than statistical mechanics for deriving important 
equations. 

Beginning the study of thermodynamics with the formal postulates, as we do in 
Part II of this book, avoids much of the complexity that nineteenth-century scientists 
had to deal with. The only drawback of the formal postulates is that they are 
rather abstract. A major purpose of Part I of this book was to make them less 
abstract by providing an explicit example of what entropy means and how it might be 
calculated. 

In the following three sections we will make the connection between the statistical 
mechanics of Part I and the thermodynamics of Part II more explicit by discussing 
the concepts of ‘state’ and ‘state function’ in the two theories. We will then present 
the postulates and laws of thermodynamics, including references to the corresponding 
results from classical statistical mechanics that were obtained in Part I. 
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9.2 Microscopic and Macroscopic States 

A microscopic state is a property of a system of particles. In classical mechanics, a 
microscopic state is characterized by specific values of the positions and momenta of 
every particle; that is, a point in phase space. In quantum mechanics, a microscopic 
state is characterized by a unique, many-particle wave function. 

In thermodynamic experiments we restrict ourselves to ‘macroscopic’ measure¬ 
ments, which are characterized by their limited resolution; by definition, macroscopic 
measurements cannot resolve individual particles or microscopic length scales. It might 
be appropriate to remember that thermodynamics was developed before scientists even 
believed that molecules existed—and they certainly did not measure the behavior of 
individual molecules. 

For both classical and quantum systems, the microscopic state is not experimen¬ 
tally accessible to macroscopic measurements. In the experiments we wish to describe, 
we can obtain some information about the microscopic state, but we cannot determine 
it completely. 

A macroscopic state is not a property of a thermodynamic system; it is a description 
of a thermodynamic system based on macroscopic measurements. Due to the exper¬ 
imental uncertainty of macroscopic measurements, a macroscopic state is consistent 
with an infinity of microscopic states. A system can be in any microscopic state that 
would be consistent with the experimental observations. 

Formally, this might seem to be a rather loose definition of what we mean 
by ‘macroscopic state’. In practice, however, there is rarely a problem. Because 
microscopic fluctuations are so much smaller than experimental uncertainties, it is 
relatively easy to specify which microscopic states are consistent with macroscopic 
measurements. 


9.3 Macroscopic Equilibrium States 

An equilibrium state is a special kind of macroscopic state. Within the limits of 
experimental measurements, the properties of a system in an equilibrium state do 
not change with time, and there is no net transfer of energy or particles into or out of 
the system. 

Equilibrium states can usually be described by a small number of measurements 
in the sense that all other properties are completely determined within experimental 
error. In the case of an ideal gas, discussed in Part I, we need to know only the number 
of particles, the volume, and the energy. Given those values, we can calculate the 
temperature, pressure, and chemical potential, along with a number of other quantities 
that we will discuss in the following chapters. 

For more complicated systems we might need to know how many molecules of 
each type are present, and perhaps something about the shape and surface properties 
of the container. In all cases, the number of quantities needed to characterize an 
equilibrium system will be tiny in comparison with the number of particles in the 
system. 
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9.4 State Functions 

The term ‘state function’ denotes any quantity that is a function of the small number 
of variables needed to specify an equilibrium state. 

The main substance of the thermodynamic postulates is that for every macroscopic 
system in thermal equilibrium there exists a state function called the entropy, which 
has certain properties listed below. Given these postulates, the entire mathematical 
structure of thermodynamics can be derived—which is the subject of the rest of the 
chapters in Part II. 

9.5 Properties and Descriptions 

In order to understand thermodynamics it will be important to keep in mind 
the distinction between a property of a system and a description of a system. 
A ‘property’ is an exact characteristic of a system. If a system contains exactly 
63749067496584764830091 particles, that is a property of the system. A ‘description’ 
is, at most, an estimate of a property. If a measurement reveals that a system has 
6.374907(2) x 10^^ particles, that is a description. 

In Chapter 4 we calculated the average number of particles in a subsystem. That 
number is an excellent description of the system for human purposes because the 
relative statistical fluctuations are so small. However, it is not the true number of 
particles at any instant of time. In a system of 10^® particles, the true number of 
particles is a property of the system, but its exact value will fluctuate over time by 
roughly 10^° particles. 

Because thermodynamics deals with descriptions of macroscopic systems, rather 
than their true properties, we will not belabor the distinction in Part II. All quantities 
will be descriptions of a system. 

9.6 Postulates of Thermodynamics 

9.6.1 Postulate 1: Equilibrium States 

Postulate 1: There exist equilibrium states of a macroscopic system that are charac¬ 
terized uniquely by a small number of extensive variables. 

Recall that extensive variables are quantities that provide a measure of the size of 
the system. Examples include the total energy, the volume, and the number of parti¬ 
cles. By contrast, intensive parameters are quantities that are independent of the size 
of the system. Examples include temperature, pressure, and chemical potential. 

The remaining postulates all specify properties of a state function called the 
‘entropy’. These properties should be compared to what you know of the entropy 
of an ideal gas from Part I. 

9.6.2 Postulate 2: Entropy Maximization 

Postulate 2: The values assumed by the exteusive parameters of au isolated composite 
system iu the abseuce of au iuterual coustraiut are those that maximize the eutropy 
over the set of all coustraiued macroscopic states. 
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The property of the entropy in the second postulate is by far the most important. 
It has two consequences that we will use repeatedly. 

First, whenever we release a constraint on a composite system, the entropy will 
increase. 


(9.1) 

This follows directly from Boltzmann’s idea that macroscopic systems go from less 
probable states to more probable states. The final state must be the most probable 
state and therefore have the highest entropy. This property is the Second Law of 
Thermodynamics. 

Eq. (9.1) can be rather puzzling. It gives time a direction, sometimes called the 
‘arrow of time’. This can seem strange, since the underlying equations of motion for 
the molecules, either classical or quantum, are time-reversal invariant. There is not 
really a contradiction, and we will reconcile the apparent conflict in Chapter 21. 

The second consequence of eq. (9.1) is that we can find the equilibrium values of 
the extensive parameters describing the amount of something after a constraint has 
been released by maximizing the entropy. This will provide an effective approach to 
solving problems in thermodynamics. 

9.6.3 Postulate 3: Additivity 

Postulate 3: The entropy of a composite system is additive over the constituent 
subsystems. 

Since we regard the composite system as fundamental in the definition of entropy, 
and the entropy of a composite system separates into the sum of the entropies of the 
subsystems, this postulate could be equally well called the postulate of separability. 
The term ‘additivity’ is generally preferred because of the concentration on the 
properties of simple systems in the history of thermodynamics. 

Most interactions between molecules are short-ranged. If we exclude gravitational 
interactions and electrical interactions involving unbalanced charges, the direct inter¬ 
action between any two molecules is essentially negligible at distances of more than a 
few nanometers. As discussed at the end of Chapter 7, this leads to an approximate 
separation of the integrals defining the entropy for the subsystems, so that the entropy 
of the composite system is just the sum of the entropies of the subsystem. 

Additivity for systems with interacting particles is an approximation in that 
it neglects direct interactions between particles in different subsystems. However, 
deviations from additivity are usually very small due to the extremely short 
range of intermolecular interactions in comparison to the size of macroscopic 
systems. 

9.6.4 Postulate 4: Monotonicity 

Postulate 4: The entropy is a monotonically increasing function of the energy for 
equilibrium values of the energy. 
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The entropy of the classical ideal gas is a monotonically increasing function of the 
energy. This monotonicity is extremely important for all systems, since it implies that 
the temperature is positive, as we have shown in Chapter 8. 

The postulate of monotonicity is usually expressed thus: ‘The entropy is a monoton¬ 
ically increasing function of the energy.’ Unfortunately, despite its importance, this 
statement is not entriely true for some of the most important models in physics, as 
we will see in Chapter 30. In practice, this blemish does not cause any inconvenience, 
because we are only interested in values of the energy that are consistent with thermal 
equilibrium, and it is true for those values. 

9.6.5 Postulate 5: Analyticity 

Postulate 5: The entropy is a continuous and differentiable function of the extensive 
parameters. 

The true number of particles is, of course, an integer. However, when we talk 
about the ‘number of particles’ or the value of N, we really mean a description of the 
number of particles that differs from the true number by less than the accuracy of our 
measurements. Since A is a description of the number of particles, it is not limited to 
being an integer. 

Thermodynamics is much easier if we can work with functions that are continuous 
and differentiable. Conveniently, this is almost always true. 

However, the postulate of analyticity can break down. When it does, it usually 
signals some sort of instability or phase transition, which is particularly interesting. 
Stability and phase transitions are discussed in Chapters 16 and 17. We will consider 
specific examples of phase transitions in Chapters 27 and 30. 

9.6.6 Postulate 6: Extensivity 

Postulate 6; The entropy is an extensive function of the extensive variables. 

‘Extensivity’ is the property that the entropy is directly proportional to the size 
of the system; if the extensive parameters are all multiplied by some number A, the 
entropy will be too. 


S{XU,XV,XN) = XS{U,V,N) (9.2) 

Mathematically, eq. (9.2) means that the entropy is assumed to be an homogeneous, 
first-order function of the extensive variables. 

The extensivity postulate is not true for all systems! 

Even such a common system as a gas in a container violates this postulate, because 
the molecules of the gas can be adsorbed onto the inner walls of the container. Since the 
surface-to-volume ratio varies with the size of the container, the fraction of molecules 
adsorbed on the inner walls also varies, and such a system is not extensive. 

For most of the book we will not assume extensivity, although in Chapter 13 we 
will see that extensivity can be extremely useful for investigating the properties of 
materials. 
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The properties of additivity and extensivity are often confused. This is probably 
due to the fact that many textbooks restrict their discussion of thermodynamics to 
homogeneous systems, for which both properties are true. Additivity is the more 
general property. It is true whenever the range of interactions between particles is 
small compared to the size of the system. Extensivity is only true to the extent that 
the surface of the system and the interface with its container can be neglected. 


9.7 The Laws of Thermodynamics 

The ‘laws’ of thermodynamics were discovered considerably earlier than the formula¬ 
tion of thermodynamics in terms of ‘postulates’. The peculiar numbering of the laws is 
due to fixing the First and Second laws before the importance of the ‘zeroth’ law was 
realized. The current numbering scheme was suggested by Ralph H. Fowler (British 
physicist and astronomer, 1889-1944) as a way of acknowledging the importance of 
the ‘zeroth’ law. 

The laws of thermodynamics are: 

Zeroth Law If two systems are each in equilibrium with a third system, they are 
also in equilibrium with each other. 

First Law Heat is a form of energy, and energy is conserved. 

Second Law After the release of a constraint in a closed system, the entropy of the 
system never decreases. 

Third Law (Nernst Postulate) The entropy of any quantum mechanical system 
goes to a constant as the temperature goes to zero. 

The Zeroth Law was derived from statistical mechanics in Section 7.8. 

The First Law is simply conservation of energy. However, at the time it was 
formulated it had just been discovered that heat was a form of energy, rather than 
some sort of mysterious fluid as had been previously supposed. 

The Second Law is essentially the same as Postulate 2 in Subsection 9.6.2. 

The Third Law is also known as the Nernst Postulate,^ adding to the confusion 
between laws and postulates. It is the only law (or postulate) that cannot be under¬ 
stood on the basis of classical statistical mechanics. For classical models, the entropy 
always goes to negative infinity as the temperature goes to zero. The explanation of 
the Third Law is intrinsically quantum mechanical. Its consequences will be discussed 
in Chapter 18, but it will not be derived until Chapter 22 in Part IV. 

In quantum statistical mechanics there is a universally recognized convention for 
the absolute value of the entropy that will be discussed in Part IV. This convention 
implies that 

^Named after the German scientist Walther Hermann Nernst (1864-1941), who in 1920 won the Nobel 
Prize for its discovery. 
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lim S{T) > 0. (9.3) 

Since the real world obeys the laws of quantum mechanics, the entropy of real systems 
obey eq. (9.3). 

The Nernst Postulate is often stated as limr^o SI{T) = 0, but that is incorrect. 
There are many examples of systems with disordered ground states that have non¬ 
zero entropy at zero temperature. 
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Perturbations of Thermodynamic 
State Functions 


When you come to a fork in the road, take it. 
Yogi Berra 


As discussed in the previous chapter, state functions specify quantities that depend 
only on the small number of variables needed to characterize an equilibrium state. It 
turns out that state functions are strongly restricted by the postulates of thermody¬ 
namics, so that we can derive a great many non-trivial equations linking observable 
quantities. The power of thermodynamics is that these relations are true for all 
systems, regardless of their composition! 


In Part I we used the symbol E to denote the energy of a system, which is customary 
in statistical mechanics. Unfortunately, it is customary in thermodynamics to use 
the symbol U for the energy. The reason for this deplorable state of affairs is that 
it is often convenient in thermodynamics to consider the energy per particle (or 
per mole), which is denoted by changing to lower case, u = U/N. Since e might 
be confused with Euler’s number, u and U have become standard. We will follow 
thermodynamics convention in Part II and denote the energy by U. It is fortunate 
that students of thermal physics are resilient, and can cope with changing notation. 


10.1 Small Changes in State Functions 

In much of thermodynamics we are concerned with the consequences of small changes. 
Part of the reason is that large changes can often be calculated by summing or 
integrating the effects of many small changes, but in many cases the experimental 
questions of interest really do involve small changes. 

Mathematically, we will approximate small changes by infinitesimal quantities. 
This, in turn, leads us to consider two distinct types of infinitesimals, depending on 
whether or not there exists a function that can be determined uniquely by integrating 
the infinitesimal. 
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10.2 Conservation of Energy 

The great discovery by James Prescott Joule (1818-1889) that heat is another form of 
energy, combined with the principle that energy is conserved, gives us the First Law 
of Thermodynamics. 

We will usually apply the First Law to small changes in thermodynamic quantities, 
so that it can be written as 


dU = dQ + aW (10.1) 

if the number of particles in the system does not change. Although dU, dQ, and dW 
are all infinitesimal quantities, there is an important distinction between those written 
with d and those written with d that is discussed in the next section. 

The change in the energy of a system during some thermal process is denoted by 
dU, and is positive when energy is added to the system. The energy added to the 
system in the form of heat is denoted by dQ, and the work done on the system is 
denoted by dW. The sign convention is that all three of these differentials are positive 
when energy is added to the system. 

Our choice of sign convention for dW is by no means universal. Taking an informal 
survey of textbooks, I have found that about half use our convention, while the 
other half take dW to be positive when work is done by the system. Be careful 
when comparing our equations with those in other books. 


In many applications of thermodynamics we will have to consider transfers of 
energy between two or more systems. In that case, the choice of what the positive 
directions for dQ and dW will be indicated by arrows in a diagram. 

10.3 Mathematical Digression on Exact and Inexact Differentials 

Consider an infinitesimal quantity defined by the equation 

dF = f{x)dx (10-2) 

where f{x) is some known function. We can always—at least in principle—integrate 
this equation to obtain a function F{x) that is unique to within an additive constant. 
Now consider the infinitesimal defined by the equation 

dF = f{x,y)dx + g{x,y)dy (10.3) 

where f{x,y) and g{x,y) are known functions. Although the differential dF can be 
integrated along any given path, the result might depend on the specific path, and 
not just on the initial and final points. In that case, no unique function F{x,y) can 
be found by integrating eq. (10.3), and dF is called an ‘inexact’ differential. The use 
of d instead of d indicates that an infinitesimal is an inexact differential. 
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Note that if two path integrals produce different values, the integral over the closed 
path going from the initial to the final point along one path and returning along the 
other will not vanish. Therefore, the failure of an integral around an arbitrary closed 
path to vanish is also characteristic of an inexact differential. 

As an example, take the following inexact differential. 


SF = xdx + xdy 


(10.4) 


Consider the integral of this differential from (0,0) to the point (1,1) by two distinct 
paths. 

Path A: From (0,0) to (x, 0) to {x,y) 

Path B: From (0,0) to (0,t/) to {x,y) 

The path integral for Path A gives 




+ xy 


and the path integral for Path B gives 



x'—O 



y'^y 


= 0+-x^ 



(10.5) 


( 10 . 6 ) 


Since the two path integrals in eqs. (10.5) and (10.6) give different results for 
y yf 0, there is no unique function F{x^y) corresponding to the inexact differential in 
eq. (10.4). 


10.3.1 Condition for a Differential to be Exact 

We can easily determine whether a differential is exact or inexact without the necessity 
of performing integrals over all possible paths. If a function F{x^ y) exists, we can write 
its differential in terms of partial derivatives. 

If the differential 

dF = f{x,y)dx + g{x,y)dy (10.8) 

is exact, we can identify f{x,y) and g{x,y) with the corresponding partial derivatives 
of F{x,y). 

= ( 10 - 9 ) 

dF 
dy 


9{x,y) 


( 10 . 10 ) 
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By looking at partial derivatives of f{x, y) and g(x, y), we find that they must satisfy 
the following condition. 


df d^F d^F dg 


( 10 . 11 ) 


dy dydx dxdy dx 


The central equality follows from the fact that the order of partial derivatives can be 
switched without changing the result for well-behaved functions, which are the only 
kind we allow in physics textbooks. 

10.3.2 Integrating Factors 

There is a very interesting and important relationship between exact and inexact 
differentials. Given an inexact differential, 


3F = f{x, y)dx + g{x, y)dy 


( 10 . 12 ) 


we can find a corresponding exact differential of the form 


dG = r{x, y) dF 


(10.13) 


for a function r(x,y), which is called an ‘integrating factor’. The function r{x,y) is 
not unique; different choices for r{x, y) lead to different exact differentials that are all 
related to the same inexact differential. 

The example given in eq. (10.4) is too easy. The function r(a;, y) = 1/a; is obviously 
an integrating factor corresponding to dG = dx + dy. 

A less trivial example is given by the inexact differential 


(10.14) 


3F = y dx + dy 


To find an integrating factor for eq. (10.14), first write the formal expression for 
the exact differential. 


dG = r{x, y) BF = r{x, y)y dx + r(x, y)dy 
The condition that dG is an exact differential in eq. (10.11) gives 


(10.15) 



(10.16) 



(10.17) 


Eq. (10.17) has many solutions, so let us restrict our search for an integrating 
factor to functions that only depend on x. 


r{x,y) = r{x) 


( 10 . 18 ) 
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Eq. (10.17) then simplifies because the partial derivative with respect to y vanishes. 


dr 

dx 


(10.19) 


Eq. (10.19) can be easily solved. 

r{x,y) = r{x) = exp(a;) 


( 10 . 20 ) 


The final result for dG is 

dG = exp(a::) dF = y exp(x) dx + exp{x)dy (10.21) 

Naturally, any constant multiple of exp(a;) is also an integrating factor. 

10.4 Conservation of Energy Revisited 

Since the energy U is a state function, the differential of the energy, dU, must be an 
exact differential. On the other hand, neither heat nor work is a state function, so 
both dQ and dW are inexact. 

10.4.1 Work 

For simplicity, first consider an infinitesimal amount of work done by using a piston to 
compress the gas in some container. Let the cross-sectional area of the piston be A, and 
let it move a distance dx, where positive dx corresponds to an increase in the volume, 
dV = Adx. The force on the piston due to the pressure P of the gas is F = PA, so 
that the work done on the gas is —Fdx or 


dW = -Fdx = -PdV (10.22) 

The volume of the system is obviously a state function, so the differential dV is 
exact. Eq. (10.22) shows a relationship between an inexact and an exact differential, 
with the function 1/P playing the role of an integrating factor. To see that dW is 
indeed an inexact differential, integrate it around any closed path in the P, E-plane. 
The result will not vanish. 

10.4.2 Heat 

The heat transferred to a system turns out to be closely related to the change in 
entropy of the system. Consider the change in entropy due to adding a small amount 
of heat dQ to the system. 

dS = S{E + dQ,V,N)-S{E,V,N)=l^^ ^ dQ=^ (10.23) 
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Again we see an equation relating exact and inexact differentials. In this case, the 
function 1/T is the integrating factor. 

Eq. (10.23) is extremely important for a number of reasons. It can be used to 
numerically integrate experimental data to calculate changes in entropy. It is also 
important for our purposes when written slightly differently. 

SQ = TdS (10.24) 

10.5 An Equation to Remember 

Eqs. (10.22) and (10.24) can be used to rewrite eq. (10.1) in a very useful form. 

dU = TdS - PdV (10.25) 

This equation is, of course, only valid when there are no leaks in the system, so that 
the total number of particles is held constant. 

Curiously enough, in Section 8.9 on the fundamental relation in the entropy 
representation, we have already derived the generalization of eq. (10.25) to include 
a change dN in the number of particles. Rewriting eq. (8.34) with dU instead of dE, 
we have 

dS = dU + dV - dN (10.26) 

Solving this equation for dU, we find one of the most useful equations in thermody¬ 
namics. 


dU = TdS - PdV + fidN (10.27) 

The reason that the differential form of the fundamental relation is equivalent to the 
First Law of Thermodynamics (energy conservation) is that energy conservation was 
used in its derivation in Part I. 


Eq. (10.27) provides another interpretation of the chemical potential /x. It is the 
change in the energy of a system when a particle is added without changing either 
the entropy or the volume. The difficulty with this explanation of the meaning of fj, 
is that it is far from obvious how to ‘add a particle without changing the entropy’. 
Apologies. 


If there is one equation in thermodynamics that you must memorize, it is eq. (10.27). 
In the following chapters you will see why this is so. 
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10.6 Problems 


Problem 10.1 

Integrating factors for inexact differentials 

We showed that the inexact differential 

SF = y dx + dy 

is related to an exact differential by an integrating function 

r{x,y) = ra;{x) 

Show that (IF is also related to another (different) exact differential by an inte¬ 
grating function of the form 

r{x,y) = ry{y) 

Derive an explicit expression for ry{y), show that the new differential is exact, and 
calculate the difference in the value of the corresponding function between the points 
(0,0) and (1,1) by two different paths, as we did in class. 
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I never failed once. It just happened to be a 2,000-step process. 
Thomas Alva Edison 


In this chapter we will discuss thermodynamic processes, which concern the conse¬ 
quences of thermodynamics for things that happen in the real world. 

The original impetus for thermodynamics, aside from intellectual curiosity, was 
the desire to understand how steam engines worked so that they could be made more 
efficient. Later, refrigerators, air conditioners, and heat pumps were developed and 
were found to be governed by exactly the same principles as steam engines. 


11.1 Irreversible, Reversible, and Quasi-Static Processes 

If we release a constraint in a composite system, the new equilibrium state maximizes 
the total entropy so that the change in entropy is non-negative for all processes in an 
isolated system. 


dS>0 (11.1) 

The change in the total entropy is zero only when the system had already been in 
equilibrium. 

When the total entropy increases during some isolated thermodynamic process, 
the process is described as ‘irreversible’. Running the process backwards is impossible, 
since the total entropy in an isolated system can never decrease. All real processes are 
accompanied by an increase in the total entropy, and are irreversible. 

On the other hand, if the change AX in some variable X is small, the change in 
entropy can also be small. Since the dependence of the entropy on a small change in 
conditions is quadratic near its maximum, the magnitude of the increase in entropy 
goes to zero quadratically, as (AX)^, while the number of changes grows linearly, 
as 1/AX. The sum of the small changes will be proportional to (AX)^/AX = AX, 
which goes to zero as AX goes to zero. Consequently, a series of very small changes 
will result in a small change of entropy for the total process. In the limit of infinitesimal 
steps, the increase of entropy can vanish. Such a series of infinitesimal steps is called 
a quasi-static process. 
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A quasi-static process is reversible. Since there is no increase in entropy, it could 
be run backwards and return to the initial state. 

The concept of a quasi-static process is an idealization, but a very useful one. The 
first applications of thermodynamics in the nineteenth century concerned the design of 
steam engines. The gases used in driving steam engines relax very quickly because of 
the high speeds of the molecules, which can be of the order of 1000 m/s. Even though 
the approximation is not perfect, calculations for quasi-static processes can give us 
considerable insight into the way real engines work. 


Quasi-static processes are not merely slow. They must also take a system between 
two equilibrium states that differ infinitesimally. The classic example of a slow 
process that is not quasi-static occurs in a composite system with two subsystems 
at different temperatures, separated by a wall that provides good but not perfect 
insulation. Equilibration of the system can be made arbitrarily slow, but the total 
entropy will still increase. Such a process is not regarded as quasi-static. 


Although the word ‘dynamics’ appears in the word ‘thermodynamics’, the theory is 
primarily concerned with transitions between equilibrium states. It must be confessed 
that part of the reason for this emphasis is that it is easier. Non-equilibrium properties 
are much more difficult to analyze, and we will only do so explicitly in Chapter 21, 
when we discuss irreversibility. 

11.2 Heat Engines 

As mentioned above, the most important impetus to the development of thermo¬ 
dynamics in the nineteenth century was the desire to make efficient steam engines. 
Beginning with the work of Sadi Carnot (French scientist, 1792-1832), scientists 
worked on the analysis of machines to turn thermal energy into mechanical energy 
for industrial purposes. Such machines are generally known as heat engines. 

An important step in the analysis of heat engines is the conceptual separation 
of the engine itself from the source of heat energy. For this reason, a heat engine is 
defined to be ‘cyclic’; whatever it does, it will return to its exact original state after 
going through a cycle. This definition ensures that there is no fuel hidden inside the 
heat engine—a condition that has been known to be violated by hopeful inventors of 
perpetual-motion machines. The only thing a heat engine does is to change energy 
from one form (heat) to another (mechanical work). 

11.2.1 Consequences of the First Law 

The simplest kind of heat engine that we might imagine would be one that takes a 
certain amount of heat SQ and turns it directly into work SW. Since we are interested 
in efficiency, we might ask how much work can be obtained from a given amount of 
heat. The First Law of Thermodynamics (conservation of energy) immediately gives 
us a strict limit. 
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aw <SQ (11.2) 

A heat engine that violated eq. (11.2) could be made into a perpetual-motion 
machine; the excess energy {aW — dQ) could be used to run the factory, while the rest 
of the work would be turned into heat and fed back into the heat engine. Because such 
a heat engine would violate the First Law of Thermodynamics, it would be called a 
Perpetual Motion Machine of the First Kind. 

Perpetual Motion Machines of the First Kind do not exist. 

A pessimist might express this result as: ‘You can’t win.’ Sometimes life seems like 
that. 

11.2.2 Consequences of the Second Law 

The limitations due to the Second Law are considerably more severe than those due 
to the First Law. 

If a positive amount of energy in the form of heat dQ is transferred to a heat engine, 
the entropy of the heat engine increases by dS = dQ/T > 0. If the heat engine now 
does an amount of work dW = dQ, the First Law (conservation of energy) is satisfied. 
However, the entropy of the heat engine is still higher than it was at the beginning 
of the cycle by an amount dQ/T > 0. The only way to remove this excess entropy so 
that the heat engine could return to its initial state would be to transfer heat out of 
the system, but this would cost energy, lowering the amount of energy available for 
work. 

A machine that could transform heat directly into work, dW = dQ, could run 
forever, taking energy from the heat in its surroundings. It would be called a Perpetual 
Motion Machine of the Second Kind because it would violate the second law of 
thermodynamics. 

Perpetual Motion Machines of the Second Kind do not exist. 

A pessimist might express this result as: ‘You can’t break even.’ Sometimes life 
seems like that, too. 

It is unfortunate that perpetual-motion machines of the second kind do not exist, 
since they would be very nearly as valuable as perpetual-motion machines of the first 
kind. Using one to power a ship would allow you to cross the ocean without the need 
of fuel, leaving ice cubes in your wake. 

11.3 Maximum Efficiency 

The limits on efficiency imposed by the First and Second laws of thermodynamics 
lead naturally to the question of what the maximum efficiency of a heat engine might 
be. The main thing to note is that after transferring heat into a heat engine, heat 
must also be transferred out again before the end of the cycle to bring the net entropy 
change back to zero. The trick is to bring heat in at a high temperature and take it 
out at a low temperature. 
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We do not need to be specific about the process, but we will assume that it is 
quasi-static (reversible). Simply require that heat is transferred into the heat engine 
from a reservoir at a high temperature Th, and heat is removed at a low temperature 
Tl < Th- If the net work done by the heat engine during an infinitesimal cycle is dW, 
conservation of energy gives us a relationship between the work done and the heat 
exchanged. 


aw = aQH+dQL (11.3) 

Sign conventions can be a tricky when there is more than one system involved. Here 
we have at least three systems: the heat engine and the two reservoirs. In eq. (11.3), 
work done by the heat engine is positive, and both dQn and both dQ^ are positive 
when heat is transferred to the heat engine. In practice, the high-temperature reservoir 
is the source of energy, so dQn > 0, while wasted heat energy is transferred to the 
low-temperature reservoir, so dQL < 0. 

Because a heat engine is defined to be cyclic, the total entropy change for a 
completed cycle must be zero. 


<iS = ff + ^ = 0 (11.4) 

The assumption of a quasi-static process (reversibility) is essential to eq. (11.4). While 
as = 0 in any case because of the cyclic nature of the heat engine, a violation of 
reversibility would mean that a third (positive) term must be added to {dQu/Tu + 
dQh/TL) to account for the additional generation of entropy. 

If we use eq. (11.4) to eliminate SQl from eq. (11.3), we obtain a relationship 
between the heat in and the work done. 


aw = 



We can now define the efficiency of a heat engine, rj. 


(11.5) 


^ = 1 _ = Th-Tl 

dQn Th Th 


( 11 . 6 ) 


The efficiency rj is clearly less than 1, which is consistent with the limitation due to 
the Second Law of Thermodynamics. Actually, the Second Law demands that the 
efficiency given in eq. (11.6) is the maximum possible thermal efficiency of any heat 
engine, whether it is reversible (as assumed) or irreversible. The proof of this last 
statement will be left as an exercise. 


11.4 Refrigerators and Air Conditioners 

Since ideal heat engines are reversible, you can run them backwards to either cool 
something (refrigerator or air conditioner) or heat something (heat pump). The 
equations derived in the previous section do not change. 
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First consider a refrigerator. The inside of the refrigerator can be represented by the 
low-temperature reservoir; the goal is to remove as much heat as possible for a given 
amount of work. The heat removed from the low-temperature reservoir is positive for 
a refrigerator, SQl > 0, and the work done by the heat engine is negative, dW < 0, 
because you have to use power to run the refrigerator. 

We can define a coefficient of performance e^j and calculate it from eqs. (11.3) and 
(11.4). 


dQL _ Tl 
-dW Th - Tl 


(11.7) 


This quantity can be much larger than 1, which means that you can remove a great 
deal of heat from the inside of your refrigerator with relatively little work; that is, you 
will need relatively little electricity to run the motor. 

An air conditioner works the same way as a refrigerator, but dQL is the heat 
removed from inside your house or apartment. The coefficient of performance is again 
given by eq. (11.7). 


The coefficient of performance for a refrigerator or an air conditioner is clearly 
useful when you are thinking of buying one. Indeed, it is now mandatory for new 
refrigerators and air conditioners to carry labels stating their efficiency. However, 
the labels do not carry the dimensionless quantity Cft.; they carry an ‘Energy 
Efficiency Ratio’ (EER) which is the same ratio, but using British Thermal Units 
(BTU) for dQL and Joules for dW. The result is that the EER is equal to cr times 
a factor of about 3.42 BTU/ J. A cynic might think that these peculiar units are 
used to produce larger numbers and improve sales—especially since the units are 
sometimes not included on the label; I couldn’t possibly comment. 


11.5 Heat Pumps 

Heat pumps also work the same way as refrigerators and air conditioners, but with 
a different purpose. They take heat from outside your house and use it to heat the 
inside of your house. The low-temperature reservoir outside your house usually takes 
the form of pipes buried outside in the yard. The goal is to heat the inside of your 
house as much as possible with a given amount of work. 

We can define a coefficient of performance for a heat pump. 


-dQn _ Th 
-dW ~ Th-Tl 


( 11 . 8 ) 


This quantity can also be significantly larger than 1, which makes it preferable to use 
a heat pump rather than running the electricity through a resistive heater. 
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11.6 The Carnot Cycle 

The Carnot cycle is a specific model of how a heat engine might work quasistatically 
between high- and low-temperature thermal reservoirs. The Carnot heat engine is 
conceived as a closed piston containing gas. The piston is brought into contact with the 
high-temperature reservoir, and the gas expands, doing work, AWh > 0. The piston 
is then removed from contact with the thermal reservoir. It now expands adiabatically, 
doing more work, AWa > 0. While it expands adiabatically, the temperature of the 
gas drops. When the temperature reaches that of the low-temperature reservoir, it is 
brought into contact with the low-temperature reservoir and isothermally compressed, 
which requires work, AWl < 0. This continues until it is removed from the low- 
temperature reservoir, when it continues to be compressed adiabatically, requiring 
more work AW'j^ < 0. The point at which the adiabatic compression begins is chosen 
carefully, so that when the temperature of the gas reaches that of the high-temperature 
reservoir it is at exactly the same volume as when the cycle began. This completes the 
Carnot cycle. 

The derivation of the efficiency of the Carnot cycle is a staple of thermodynamics 
books, but I think that it is more fun (and more useful) to do it yourself. The answer 
is, of course, the same as we found with less effort in eq. (11.6). 

11.7 Problems 


Problem 11.1 

Efficiency of real heat engines 

We showed that the efficiency of an ideal heat engine is given by 


Th-Tl , Tl 

jyideal — „ — ^ rri 


Real heat-engines must be run at non-zero speeds, so they cannot be exactly quasi¬ 
static. Prove that the efficiency of a real heat engine is less than that of an ideal heat 
engine. 


^ ^ ^ideal 


Problem 11.2 

Maximum work from temperature differences 

Suppose we have two buckets of water with constant heat capacities Ca and Cb, so 
that the relationship between the change in temperature of bucket A and the change 
in energy is 


dUA = CAdT 
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with a similar equation for bucket B. 

The buckets are initially at temperatures q and Tb,o- 

The buckets are used in conjunction with an ideal heat engine, guaranteed not to 
increase the world’s total entropy (FBN Industries, patent applied for). 

1. What is the final temperature of the water in the two buckets? 

2. What is the maximum amount of work that you can derive from the heat energy 
in the buckets of water? 

3. If you just mixed the two buckets of water together instead of using the heat 
engine, what would be the final temperature of the water? 

4. Is the final temperature in this case higher, lower, or the same as when the heat 
engine is used? Explain your answer. 

5. What is the change in entropy when the water in the two buckets is simply mixed 
together? 


Problem 11.3 

Work from finite heat reservoirs 

1. Suppose we have N objects at various initial temperatures. The objects have 
constant but different heat capacities {Cj | j = 1,..., N}. The objects are at initial 
temperatures {Tj,o|j = 1, ■ • ■ j N}. 

If we again have access to an ideal heat-engine, what is the maximum work 
we can extract from the thermal energy in these objects? 

What is the final temperature of the objects? 

2. Suppose that the heat capacities of the objects were not constant, but propor¬ 
tional to the cube of the absolute temperature. 

{C,iT)=A,T^\j = l,...,N} 
where the Aj are constants. 

What is the maximum work we can extract from the thermal energy in these 
objects? 

What is the final temperature of the objects? 
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Nobody knows why, but the only theories which work are the mathematical 
ones. 

Michael Holt, in Mathematics in Art 


Although the fundamental relation in either the entropy, S = S{U,V, N), or energy, 
U = U{S,V, N), representations contains all thermodynamic information about the 
system of interest, it is not always easy to use that information for practical calcula¬ 
tions. 

For example, many experiments are done at constant temperature, with the 
system in contact with a thermal reservoir. It would be very convenient to have 
the fundamental relation expressed in terms of the intensive parameters T and P, 
instead of the extensive parameters S and V. It turns out that this can be done, but 
it requires us to introduce new functions that are generally known as thermodynamic 
potentials. The mathematics required involves Legendre transforms (named after the 
French mathematician Adrien-Marie Legendre (1752--1833)), which will be developed 
in the next section. The rest of the chapter is devoted to investigating the properties 
and advantages of various thermodynamic potentials. 

12.1 Mathematical digression: the Legendre Transform 

Before we go through the details of the Legendre transform, it might be useful to 
discuss why we need it and what problems it solves. 

12.1.1 The Problem of Loss of Information 

The basic structure of representing the fundamental relation in terms of T, P, or fj,, 
instead of U, V, or N, is that we want to use the derivative of a function as the 
independent variable. For simplicity, consider some function 

y = y{x) (12.1) 


and its derivative 



p{x) 


( 12 . 2 ) 
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Fig. 12.1 Illustration of distinct functions y = y{x) that have the same slope at a given value 

ofy- 

We could, of course, invert eq. (12.2) to find x = x{p), and then find y = y{x{p)) = 
y{p). Unfortunately, this procedure results in a loss of information. The reason is 
illustrated in Fig. 12.1, which shows that all functions of the form y{x — Xo) give 
exactly the same function y(j)). If we only have y(j)), we have lost all information 
about the value of Xo- 


12.1.2 Point Representations and Line Representation 

To solve the problem of lost information, we can turn to an alternative representation 
of functions. 

When we write y = y{x), we are using a ‘point’ representation of the function; for 
each value of x, y(x) specifies a value of y, and the two together specify a point in the 
a;,?/-plane. The set of all such points specifies the function. 

However, we could also specify a function by drawing tangent lines at points along 
the curve, as illustrated in Fig. 12.2. The envelope of the tangent lines also reveals the 
function. This is known as a ‘line’ representation of a function. 


12.1.3 Direct Legendre Transforms 

The idea of a Legendre transform is to calculate the equations of the tangent lines 
that carry the full information about the function. The information we need for each 
tangent line is clearly the slope, which we want to be our new variable, and the y- 
intercept. 

Consider a function, y = y{x), which is plotted as a curve in Fig. 12.3. We can 
construct a straight-line tangent to any point (x, y) along the curve, which is also 
shown in Fig. 12.3. If the y-intercept is located at ( 0 ,( 7 ), we can calculate the slope of 
the tangent line. 



y-q 

X — 0 


p{x) 


(12.3) 
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Fig. 12.2 Illustration of how to represent a function by the envelope of tangent lines. 



Fig. 12.3 Graphical representation of a general Legendre transform. The straight line is tangent 
to the curve y = y{x). The y-mtercept of the tangent line is located at (0,q). 


Assuming that p = p(x) is monotonic, we can invert it to obtain x = x(p). We can 
then eliminate x in favor of p in eq. (12.3) and solve for q = q{p). 


q = y-px = q{p) = y[p] 


(12.4) 


The square brackets in y[p\ indicate that y[p\ = y — px is the Legendre transform of 
y{x), with p as the new independent variable. 
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Table 12.1 General form of 

a Legendre transform. 

Direct transform 

Inverse transform 

y = y{x) 
dy = pdx 

q = q{p) 

dq = —X dp 

dy 

P=Tx 

q = y-px 

q = q{p) = y[v] 

dq 

dp 

y = q + xp 

y = y{x) 

dq = —X dp 

dy = pdx 


The Legendre transform and its inverse are summarized in Table 12.1. 

12.1.4 Inverse Transformations 

The inverse Legendre transformation proceeds in the same way. In this case, 

x = -^=x{p) (12.5) 

dp 

Assuming that x = x{p) is monotonic, we can invert it to obtain p = p{x). Eq. (12.4) 
can be solved for y. 

y = q — {—x)p = q + xp (12.6) 

We can then eliminate p in favor of x in eq. (12.3) and solve for y = y{x). There is no 
loss of information in either the direct or inverse Legendre transform. 

12.1.5 Legendre Transform of Infinitesimals 

It will be very useful to write the information in eqs. (12.3) and (12.5) using infinites¬ 
imals. 


dy = pdx (12-7) 

dq = —X dp (12-8) 

Note that going between the original infinitesimal and the Legendre transform merely 
involves switching x and p and changing the sign. This simple observation will make 
life much easier when working with the thermodynamic potentials discussed in the 
rest of this chapter—assuming that you have memorized eq. (10.27). 

12.2 Helmholtz Free Energy 

Many experiments are carried out at constant temperature, and we have thermometers 
to measure the temperature. However, we do not have an easy way of measuring 
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entropy, so it would be very useful to express the fundamental relation, which contains 
all thermodynamic information about a system, in terms of the temperature. 

We begin with the fundamental relation in the energy representation, U = 
U{S,V, N). We wish to replace the entropy S with the temperature T as the indepen¬ 
dent variable. 



We follow exactly the same procedure as in Section 12.1, except that we use partial 
derivatives and hold V and N constant. The Legendre transform of the energy with 
respect to temperature is called the Helmholtz free energy, and is denoted by the 
symbol F in this book, although you will also see it called A in some other texts. 

F{T,V,N) = U[T] = U-TS (12.10) 

Since you have memorized the differential form of the fundamental relation in the 
energy representation, eq. (10.27), 

dU = TdS - PdV + fidN (12.11) 

you can easily find the differential form of the fundamental relation in the Helmholtz 
free energy representation by exchanging S and T in the first term and reversing the 
sign. 


dF = -SdT - PdV + ndN 


( 12 . 12 ) 


From eq. (12.12) we can easily read off the partial derivative of F with respect to the 
new independent variable T. 


S = - 



(12.13) 


The other partial derivatives look much the same as before, but it is important to 
keep track of what is being held constant. For example, in the energy representation, 
the pressure is found from the partial derivative 


P = - 



(12.14) 


where the entropy is held constant. In the Helmholtz free energy representation, the 
pressure is found from a similar partial derivative 


P=- 



(12.15) 


but now the temperature is held constant. 

The Legendre transforms, direct and inverse, for the Helmholtz free energy are 
summarized in Table 12.2. 


128 


Thermodynamic Potentials 


Table 12.2 Helmholtz free energy. 


Direct transform 

Inverse transform 

U = U{S, V, N) 

F = F{T, V, N) 

dU = TdS - PdV + (idN 

dF = -SdT - PdV + tidN 

fdU\ 

r. fdF\ 

T={ 

—S = 1 —— 1 

\ ) V,N 

\ / V,JV 

F = U-TS 

P = F + TS 

F = F{T, V, N) = U[T] 

u = U{S, V, N) 

dF = -SdT - PdV + tidN 

dU = TdS - PdV + ndN 


12.3 Enthalpy 

The Legendre transform of the energy with respect to pressure is called the enthalpy, 
and is denoted by the symbol H = U\P]. It is often convenient because it takes the 
pressure as one of the independent variables. It is very widely used in chemistry for 
that reason, since most chemical experiments are carried out at atmospheric pressure. 

We again begin with the fundamental relation in the energy representation, 
U = U{S,V, N). This time we will replace the volume V with the pressure P as the 
independent variable. 


P = - 



(12.16) 


To find the enthalpy we follow exactly the same procedure as in the previous 
section. 


H{S,P,N) = U[P] = U + PV (12.17) 

The differential form for the enthalpy is found from the differential form of the 
fundamental relation in the energy representation, eq. (12.11), but we now exchange 
V and P and change the sign of that term. 


dH = TdS + VdP + yidN 


(12.18) 


From eq. (12.18) we can read off the partial derivative of H with respect to the new 
independent variable T. 


V = 



(12.19) 


The Legendre transforms, direct and inverse, for the enthalpy are summarized in 
Table 12.3. 

In some chemistry texts, the enthalpy is also referred to as the ‘heat content’, 
even though dQ is not an exact differential, and we cannot properly speak of the 
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Table 12.3 

Enthalpy. 

Direct transform 

Inverse transform 

U = U{S, V, N) 

H = H{S, P, N) 

dU = TdS - PdV + fidN 

dH = TdS + VdP + pdfV 

P = -(-) 

\dv)s,N 

y = i-) 

V ) S,N 

H = U + PV 

U = H - PV 

H = H{S, V, N) = U[P] 

U = U{S, V, N) 

dH = TdS + VdP + fidN 

dU = TdS - PdV + ndN 


amount of ‘heat’ in a system. The reason for this usage can be seen by referring to 
eq. (12.18). Most chemical experiments are carried out under conditions of constant 
pressure, with a fixed number of particles. If we recall the relationship between heat 
transfer and entropy change in eq. (10.24), we find that the change in enthalpy is equal 
to the heat transferred into the system under these conditions. 

dH = TdS + VdP + /idiV (12.20) 

= TdS + 0 + 0 
= dQ 

This equation is a bit peculiar in that it sets an exact differential equal to an inexact 
differential. This came about by specifying the path of integration with the conditions 
dP = 0 and dN = 0, which made the path one-dimensional. Since one-dimensional 
differentials are always exact, dQ = dH along the specified path. 

12.4 Gibbs Free Energy 

The Gibbs free energy is the Legendre transform of the energy with respect to both the 
temperature and the pressure. It is denoted with the letter G, which seems remarkably 
logical after seeing the standard choices for the Helmholtz free energy {F or A) and 
the enthalpy [H). There are three ways to calculate it: 

1. Find the Legendre transform of the Helmholtz free energy with respect to 
pressure. 

2. Find the Legendre transform of the enthalpy with respect to temperature. 

3. Start with the energy and carry out both Legendre transforms at the same time. 

Since the first two possibilities should be self evident, I will just give the third option 
in the form of Table 12.4. 
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Table 12.4 

Gibbs free energy. 

Direct transform 

Inverse transform 

U = U{S, V, N) 

dU = TdS - PdV + pdN 

G = G(r, P, N) 

dG = -SdT + VdP + pdN 

T=(-) 

\ ) V,N 

\dv)s,N 
G=U-TS + PV 

G = G(r,P,/V) = U[T,P] 

--m.. 

-(S)„ 

U = G + TS-PV 
u = U{S, V, N) 

dG = -SdT + VdP + pdN 

dU = TdS - PdV + pdN 


12.5 Other Thermodynamic Potentials 

Since there are three thermodynamic variables for even the simplest system (U, V, 
and N), and we can construct a Legendre transform from any combination of the 
corresponding partial derivatives, there are 2^ = 8 different thermodynamic potentials. 
Each of these potentials provides a representation of the fundamental relation. They 
can all be derived by the same procedures that we used for F, iJ, and G. 

12.6 Massieu Functions 

In addition to those discussed above, there is another class of thermodynamic 
potentials, known as Massieu functions, which are generated from the fundamental 
relation in the entropy representation, S = S{U,V, N). The differential form for the 
fundamental relation in the entropy representation was found in eq. (10.26). 

(l)diV (12.21) 

As can be seen from eq. (12.21), the natural variables for a Legendre transform are 
1/T, P/T, and p/T. Although these variables appear rather strange, working with 
them is essentially the same as for the more usual thermodynamic potentials. 

It might be easier to think about them with a change of notation. For example, we 
might make (3 = l/ksT the independent variable in the transformed potential. S')/?]. 

12.7 Summary of Legendre Transforms 

The usefulness of Legendre transforms and thermodynamic potentials is not limited to 
providing alternative representations of the fundamental relation, although that is very 
important. We will see in Chapter 14 that they also play a crucial role in deriving ther¬ 
modynamic identities; that is, equations relating measurable quantities that are true 
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for all thermodynamic systems. In Chapter 15 we will see that thermodynamic poten¬ 
tials can be used to determine equilibrium conditions from extremum principles similar 
to the maximization principle for the entropy. Finally, in Chapter 16 we will find that 
they are essential to deriving stability criteria that are also valid for all systems. 


12.8 Problems 


Problem 12.1 

Legendre transform of the energy with respect to temperature for the 
classical ideal gas 

Starting with the fundamental equation in the entropy representation (that is, the 
entropy of the classical ideal gas—which you have memorized): 

5 = S{U, V, N) = ksN 0 In + In + In 

1. Derive the fundamental relation in the energy representation ([/ = U{S,V, N)). 

2. Derive the temperature as a function of entropy, volume, and number of particles 
(one of the three equations of state). 

3. Find the entropy as a function of temperature, volume and number of particles. 
Is this an equation of state or a fundamental relation? 

4. Derive the Helmholtz free energy of the classical ideal gas, F{T, V, N) = 
U -TS. 


Problem 12.2 

More Legendre transforms 

1. Starting with the fundamental equation in the Helmholtz free energy representa¬ 
tion {F = F{T,V, N)) that you derived for the previous assignment, derive the 
fundamental relation in the Gibbs free energy representation (G = G{T, P, N)). 

2. Find the volume of the ideal gas as a function of temperature, pressure, and 
number of particles by taking a derivative of the Gibbs free energy. 

Problem 12.3 

General thermodynamic function 

Suppose we know some terms in a series expansion of the Gibbs free energy as a 
function of T and P for some region near the point (To,Po)- 

G = AT + BT^ + CP + DP^ + FTP 
where A, H, G, D, and E are constants. 


132 


Thermodynamic Potentials 


1. Find the volume of the system as a function of T and P. 

2. The isothermal compressibility is defined as 


kt 


1 

V 



T,N 


Find Kt for this system. 

3. Find the entropy as a function of T and P for this system. 


Problem 12.4 
Enthalpy 

The enthalpy of a material in a certain range of temperature and pressure is well 
approximated by the expression: 

H = A + BT+CP + DT^ + EP^ + FTP 

1. Is this expression for H an approximation to the fundamental relation, or is it 
an equation of state. Explain your answer for credit. 

2. Calculate the specific heat at constant pressure, cp, within the approximation 
for H given above. 


13 

The Consequences of Extensivity 


Real knowledge is to know the extent of one’s ignorance. 
Confucius 


In this chapter we return to the thermodynamic postulates and consider the conse¬ 
quences of extensivity. As defined in Section 9.6.6, the entropy is extensive if 

S{XU,XV,XN) = XS{U,V,N) (13.1) 

for any positive value of the parameter A. The postulate of extensivity means that S, 
U, V, and N are all proportional to the size of the system. 

It is common for books on thermodynamics to assume from the beginning that all 
systems are homogeneous. If a system is homogeneous, then additivity automatically 
implies extensivity. As noted in Subsection 9.6.6, because of this, the properties of 
additivity and extensivity are often confused. However, they are not the same. 

As long as the molecular interactions are short-ranged, the entropy of the macro¬ 
scopic system will be additive. Nevertheless, most real systems are not extensive. A 
simple example is given by a container of gas that can adsorb molecules on its walls. As 
the size of the container is varied, the surface-to-volume ratio varies, and the fraction 
of the molecules adsorbed on the walls also varies. The properties of the system will 
depend on the surface-to-volume ratio, and the entropy will not be extensive. 

Even crystals or liquids with free boundary conditions will have contributions to 
the energy and free energies from surfaces and interfaces. It is important to be able 
to treat such systems, so that surface properties can be unambiguously defined and 
studied. 

On the other hand, we are often interested in the bulk properties of a material and 
would like to investigate its thermodynamic behavior without concerning ourselves 
with surfaces or interfaces. In these circumstances it is reasonable to consider an 
homogeneous and extensive system consisting entirely of the material of interest. Such 
a system would satisfy the postulate of extensivity. 

13.1 The Euler Equation 

If a system is extensive its energy is an homogeneous first-order function, and satisfies 
eq. (13.1), XU{S,V,N) = U{XS, XV, XN), for any value of A. 
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If we differentiate eq. (13.1) with respect to A, we find 



_ dUjXS, XN) djXS) 


d{XS) dX 




(13.2) 




(13.3) 


If we now substitute for the partial derivatives, we find 


U = TS - PV + fiN 


(13.4) 


which is known as the Euler equation. 

Note that the Euler equation is trivial to remember if you have memorized 
eq. (10.27), which states: dU = TdS — PdV + ^dN. 

The Euler equation can also be expressed in terms of the entropy. 



(13.5) 


This equation can be found directly from the homogeneous first-order property of the 
entropy given in eq. (13.1), or more simply by rewriting eq. (13.4). 

The most important consequence of extensivity is the Euler equation. It can be 
extremely useful, but it must be kept in mind that it is valid only for extensive systems. 

13.2 The Gibbs-Duhem Relation 

If a system is extensive, the three intensive parameters, T, P, and fi are not inde¬ 
pendent. The Gibbs-Duhem relation, which can be derived from the Euler equation, 
makes explicit the connection between changes in these parameters. 

First, we can write the complete differential form of eq. (13.4). 


dU = TdS - PdV -k ftdN + SdT - VdP + Nd^i 


(13.6) 


Comparing eq. (13.6) to the differential form of the First Law, eq. (10.27) 


dU = TdS - PdV + fidN 


( 13 . 7 ) 
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and subtracting the one from the other, we find the Gibbs-Duhem relation. 

0 = SdT - VdP + Nd^i (13.8) 


Another way of writing this is to exhibit the change in /r as a function of the changes 
in T and P. 


d^ = — 




(13.9) 


These forms of the Gibbs-Duhem relation involve the entropy per particle and 
the volume per particle, which might not be the most convenient quantities to work 
with. If we carry out the same derivation starting with the fundamental relation in 
the entropy representation, S = S(U,V, N), we find an alternative formulation of the 
Gibbs-Duhem relation. 


d 



(13.10) 


For our simple example of a one-component system, we can see that only two 
parameters are free. For any changes in the temperature or pressure, the change in the 
chemical potential is fixed by eq. (13.9) or (13.10). The number of free parameters 
needed to specify a thermodynamic system is called the number of thermodynamic 
degrees of freedom for the system. A simple, one-component system has two degrees 
of freedom. 

The Euler equation and the Gibbs-Duhem relation can both be generalized to r 
components by including a chemical potential term each component j in 

the differential form of the fundamental relation. 


dU = TdS - PdV (13.11) 

i=i 

The Gibbs-Duhem relation becomes 

r 

0 = SdT-VdP + '^NjdFj (13.12) 

1=1 


A direct consequence is that if we have an extensive system with r components, it will 
have r -|- 1 thermodynamic degrees of freedom. 


13.3 Reconstructing the Fundamental Relation 

We have seen in Section 8 that although an equation of state contains thermodynamic 
information, the information is not complete. For a general system we can only 
reconstruct the fundamental relation and obtain complete information if we know 
all three equations of state for a one-component system, or all r -|- 2 equations of state 
for a system with r components. 
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An important consequence of the Gibbs-Duhem relation is that for extensive 
systems we only need r + 1 equations of state for an r-component system to recover 
the fundamental relation and with it access to all thermodynamic information. 

As an example, consider the classical, one-component ideal gas. Suppose we were 
to know only two of the three equations of state. 

PV = NkBT (13.13) 

and 

U=^NkBT (13.14) 

If we can calculate the third equation of state, we can substitute it into the Euler 
equation to find the fundamental relation. 

To carry out this project, it is convenient to introduce the energy per particle 
u = U/N and the volume per particle v = V/N. The ideal gas law in the form 

^ = kBV~^ (13.15) 


gives us 


and the energy equation 


gives us 



= —kBV ^dv 




3 

2 


kBU ^du 


(13.16) 


(13.17) 


(13.18) 


Inserting eqs. (13.16) and (13.18) into the Gibbs-Duhem relation, eq. (13.10), 

^(t) ^ (0 (13.19) 

we find 

d (^) = —u-kBU~^du — vkBV~^dv = —-kBU~^du — kBV~^dv (13.20) 
Integrating eq. (13.20) we obtain the third equation of state. 

^ - ln(M) - kB ln(z;) -|- X 


(13.21) 
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An arbitrary constant of integration, X, is included in this equation, which reflects 
the arbitrary constant in classical expressions for the entropy. Putting these results 
together to And a fundamental relation is left as an exercise. 


13.4 Thermodynamic Potentials 

The Euler equation eq. (13.4) puts strong restrictions on thermodynamic potentials 
introduced in Chapter 12 for extensive systems in the form of alternative expressions. 
For the thermodynamic potentials F, H, and G, we have the following identities for 
extensive systems. 


F =U -TS= -PV + /iiV 

(13.22) 

H = U + PV = TS + ^iN 

(13.23) 

G = U -TS + PV = ^iN 

(13.24) 


Note that eq. (13.24) expresses the chemical potential as the Gibbs free energy per 
particle for extensive systems. 

(13.25) 

The thermodynamic potential U[T,P,^] vanishes for extensive systems. 

U[T,P,fi] = U-TS + PV - ^lN = 0 (13.26) 

For this reason, ?7[T, P, is often omitted in books on thermodynamics that restrict 
the discussion to extensive systems. However, C/[T, P, /i] does not vanish for general 
thermodynamic systems, and it is sensitive to surface and interface properties. 
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Thermodynamic Identities 


All correct reasoning is a grand system of tautologies, but only God can make 
direct use of that fact. 


Herbert Simon, political scientist, economist, and psychologist 
(1916-2001), Nobel Prize in Economics, 1978 


14.1 Small Changes and Partial Derivatives 

Many of the questions that arise in thermodynamics concern the effects of small 
perturbations on the values of the parameters describing the system. Even when 
the interesting questions concern large changes, the best approach to calculations 
is usually through adding up a series of small changes. For these reasons, much of 
thermodynamics is concerned with the response of a system to small perturbations. 
Naturally, this takes the mathematical form of calculating partial derivatives. 

Ultimately, the values of some partial derivatives must be found either from 
experiment or from the more fundamental theory of statistical mechanics. However, the 
power of thermodynamics is that it is able to relate different partial derivatives through 
general identities that are valid for all thermodynamic systems. In this chapter we will 
develop the mathematical tools needed to derive such thermodynamic identities. 


14.2 A Warning about Partial Derivatives 

For much of physics, partial derivatives are fairly straightforward: if we want to 
calculate the partial derivative with respect to some variable, we treat all other 
variables as constants and take the derivative in the same way. 

However, in thermodynamics it is rarely obvious what the ‘other’ variables are. 
Consequently, it is extremely important to specify explicitly which variables are being 
held constant when taking a partial derivative. 

For example, consider the innocent-looking partial derivative, dU/dV, for a clas¬ 
sical ideal gas. If we hold T constant. 



(14.1) 
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but if we hold P constant, 



P,N 


d_ 

W 




(14.2) 


When S is held constant we can use the equation dU = TdS — PdV to see that 




S,N 


= -P 


(14.3) 


We find three different answers—one positive, one negative, and one zero—^.just from 
changing what we are holding constant! 

Don’t forget to write explicitly what is being held constant in a partial derivative! 


14.3 First and Second Derivatives 


Since you memorized eq. (10.27) long ago, you know the first derivatives of the 
fundamental relation in the energy representation. 


/m 

m 


dU 


V,N 


S,N 


s,v 


T 

(14.4) 

-P 

(14.5) 


(14.6) 


Since you also know how to find the Legendre transform of the differential form of 
a fundamental relation, you know the first derivatives of the Gibbs free energy. 



V,N 

S,N 

S,V 


-S 

(14.7) 

V 

(14.8) 

F 

(14.9) 


Essentially, we find various subsets of U, S, T, V, P,fi, and N when we take a first 
partial derivative of the fundamental relation. Second derivatives provide information 
about how these quantities change when other quantities are varied. 

There are clearly many different second derivatives that can be constructed from all 
the variables we have introduced—especially when we remember that holding different 
quantities constant produces different partial derivatives. However, the number of 
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independent second derivatives is limited. This is the fundamental reason for the 
existence of thermodynamic identities. 

If we consider the fundamental relation in the energy representation for a simple 
system with a single component, first derivatives can be taken with respect to the 
three independent variables: S, V, and N. That means that there are a total of six 
independent second derivatives that can be formed. All other second derivatives must 
be functions of these six. 

In many applications the composition of the system is fixed, so that there are 
only two independent variables. This means that even if we are dealing with a multi- 
component system, there are only three independent second derivatives. For example, 
if we consider U{S, V, N) with N held constant, the independent second derivatives 
are: 



f d'^U \ / d^U \ 

\dSdv) ~ \dVds) 


and 


d^U\ 


(14.10) 

(14.11) 


(14.12) 


This is a very important observation, because it implies that only three measure¬ 
ments are needed to predict the results of any experiment involving small changes. 
This can be extremely important if the quantity you want to determine is very difficult 
to measure directly. By relating it to things that are easy to measure, you can both 
increase accuracy and save yourself a lot of work. 

On the other hand, knowing that everything is related is not the same as knowing 
what that relationship is. The subject of thermodynamic identities is the study of how 
to reduce any partial derivative to a function of a convenient set of standard partial 
derivatives. 


14.4 Standard Set of Second Derivatives 

The three standard second derivatives are: 

The coefficient of thermal expansion 

(14.13) 

(14.14) 

T,N 


1 fdV 


P,N 


The isothermal compressibility 


1 fdV 
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The minus sign in the definition is there to make kt positive, since the volume 
decreases when pressure increases. 

The specific heat per particle at constant pressure 


T /dS\ 

The specific heat per particle at constant volume 


(14.15) 


T /dS\ 


(14.16) 


Well, yes, 1 have listed four of them. 

However, because there are only three independent second derivatives, we will be 
able to find a universal equation linking them. Usually, the first three are the easiest to 
measure, and they are taken as fundamental. Trying to measure cy for iron is difficult, 
but cp is easy. However, measuring cy for a gas might be easier than measuring cp, 
so it has been included. 

If we are interested in the properties of a particular system we might also use the 
‘heat capacity’, denoted by a capital U; that is, Cp = Ncp and Cy = Ncy- 

It should be noted that 1 have defined cp and cy as specific heats per particle. It 
is also quite common to speak of the specific heat per mole, for which division by the 
number of moles would appear in the definition instead of division by N. It is also 
sometimes useful to define a specific heat per unit mass. 

Our specific goal for the rest of the chapter is to develop methods for reducing all 
possible thermodynamic second derivatives to combinations of the standard set. 

The four second derivatives listed in eqs. (14.13) through (14.16) should be mem¬ 
orized. While they are not as important as eq. (10.27), it is useful not to have 
to consult them when deriving thermodynamic identities. The brief time spent 
memorizing them will be repaid handsomely. 


14.5 Maxwell Relations 

The first technique we need is a direct application of what you learned about exact dif¬ 
ferentials in Section 10.3.1. Since every differential representation of the fundamental 
relation is an exact differential, we can apply eq. (10.11) to all of them. 

For example, start with the differential form of the fundamental relation in the 
energy representation, eq. (10.27), which you have long since memorized. 

dU = TdS - PdV + fidN ( 14 . 17 ) 

If N is held constant, this becomes 


dU = TdS - PdV 


( 14 . 18 ) 
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Applying the condition for an exact differential, eq. (10.11), produces a new identity. 


/dT 


S,N 



(14.19) 


Eq. (14.19) and all other identities derived in the same way are called Maxwell 
relations. They all depend on the condition in eq. (10.11) that the differential of a 
thermodynamic potential be an exact differential. 

The thing that makes Maxwell relations so easy to use is that the differential 
forms of the fundamental relation in different representations are all simply related. 
For example, to find dF from dU, simply switch T and S in eq. (14.17) and change 
the sign of that term. 


dF = -SdT - PdV + fidN 


From this equation, we find another Maxwell relation. 


_ _ /dP 


(14.20) 


(14.21) 


It is even easy to find exactly the right Maxwell relation, given a particular partial 
derivative that you want to transform, by finding the right differential form of the 
fundamental relation. 

For example, suppose you want a Maxwell relation to transform the following 
partial derivative. 


(?L 

\J^ 


s,n 



(14.22) 


Begin with eq. (14.17) and choose the Legendre transform that 

• leaves T in front of dS, so that T is being differentiated; 

• changes —PdV to VdP, so that the derivative is with respect to P; and 

• changes fidN to —Nd^, so that p is held constant. 

The result is the differential form of the fundamental equation in the U[P,fi] repre¬ 
sentation. 


dU[P, fi] = TdS + VdP + Ndfi 


Now apply eq. (10.11) with dfi = 0. 

/dT 


S,fl 



(14.23) 


(14.24) 


Note that you do not even have to remember which representation you are 
transforming into to find Maxwell relations. The thermodynamic potential U[P,fi] 
is not used sufficiently to have been given a separate name, but you do not need to 
know its name to derive the Maxwell relation in eq. (14.24). 
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You do have to remember that the Legendre transform changes the sign of the 
differential term containing the transformed variables. But that is easy. 


14.6 Manipulating Partial Derivatives 

Unfortunately, Maxwell relations are not sufficient to derive all thermodynamic iden¬ 
tities. We will still need to manipulate partial derivatives to put them into more 
convenient forms. A very elegant way of doing that is provided by the use of Jacobians. 

We do have to introduce some new mathematics at this point, but the result will 
be that thermodynamic identities become quite easy to derive—perhaps even fun. 


14.6.1 Definition of Jacobians 

Jacobians are defined as the determinant of a matrix of derivatives. 


d (m, v) 
d{x,y) 


Since we have only the two variables, x and y, in this definition, I have suppressed 
the explicit indication of which variables are being held constant. This is an exception 
that should not be imitated. 

The definition in eq. (14.25) can be extended to any number of variables, but 
there must be the same number of variables in the numerator and denominator; the 
determinant must involve a square matrix. 


ou ou 

dx dy 
dv dv 

dx dv 


du dv 
dx dy 


du dv 
dy dx 


(14.25) 


14.6.2 Symmetry of Jacobians 

The Jacobian changes sign when any two variables in the numerator or the denom¬ 
inator are exchanged, because the sign of a determinant changes when two rows or 
two columns are switched. (Variables cannot be exchanged between numerator and 
denominator.) 


d (u, ^) d{v ,u) d{v,u) d{u,v) 

d{x,y) d{x,y) d{y,x) d{y,x) 

This also works for larger Jacobians, although we will rarely need them. 


(14.26) 


d{u,v,w,s) d{v,u,w,s) d{v^u,w,s) 

d{x,y,z,t) d{x,y,z,t) d{y,x,z,t) 


(14.27) 
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14.6.3 Partial Derivatives and Jacobians 

Consider the Jacobian 


Since 


and 


we have 


d{u,y) 

d{x,y) 


du du 

dx dy 

^ % 

dx dy 



dy^ 

dy 


d{u,y) ^ 
d{x,y) \dxj 


(14.28) 


(14.29) 


(14.30) 


(14.31) 


This is the link between partial derivatives in thermodynamics and Jacobians. 
We will also see it in an extended form with more variables held constant. 


For example: 


d{u,y,z) ^ 
d{x,y,z) \dx)y^^ 


d{V,T,N) fdV\ 

~ \dP) t,n 


(14.32) 


(14.33) 


14.6.4 A Chain Rule for Jacobians 

The usual chain rule for derivatives takes on a remarkably simple form for Jacobians. 


d{u,v) d{u,v) d{r,s) 
d{x,y) d{r,s) d{x,y) 


(14.34) 


To prove this, it is convenient to introduce a more compact notation. 



X 


(14.35) 
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Now we can write 

d{u,v) d{r,s) 
d{r,s) d{x,y) 


which proves eq. (14.34). 


Uj' Ug 


Tx Ty 

Vr Vs 


Sx Sy 


UrTX ~\~ UgSx Ur^y UgSy 
Vj-Tx ~\~ '^s’^x '^s^y ~\~ UgSy 

'^x ^y 
Vx Vy 

d (m, v) 
d{x,y) 


(14.36) 


14.6.5 Products of Jacobians 


The following identities are very useful. 

d{u,v) d{a,b) d{u,v) d{a,b) d{a,b) d{u,v) 
d{x,y) d{c,d) d{c,d) d{x,y) d{x,y) d{c,d) 


(14.37) 


To prove them, we use eq. (14.34) which we showed to be an identity in the previous 
subsection. 

d{u,v) d{a,b) ^ d{u,v) d{r,s) d{a,b) d{r,a) ('14 381 

d{x,y) d{c,d) d{r,s) d{x,y) d (r.s) d{c,d) 

d{u,v) d{r,s) d{a,b) d{r,s) 

d{r,s) d{c,d) d (r.s) d{x,y) 

d{u,v) d{a,b) 
d{c,d) d{x,y) 


For many purposes, eq. (14.37) allows expressions of the form d{x,y) to be 
manipulated almost as if they were algebraic factors. 


14.6.6 Reciprocals of Jacobians 

Reciprocals of Jacobians are both simple and useful. 

Write the identity in eq. (14.34) with {x^y) set equal to {u,v). 


d{u,v) d{u,v)d(r,s) 
d{u,v) d{r,s)d{u,v) 

But 


(14.39) 


d {u, v) 

Uu V>y 


1 0 

d {u, v) 

Vu Vy 


0 1 


(14.40) 
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SO that 


d{u,v) ^ / d (r, s) 

d{r,s) / d{u,v) 


(14.41) 


Using eq. (14.41) for reciprocals, we can express the chain rule from the previous 
section in a particularly useful alternative form. 


d{u,y) ^ d{u,y) j d (x, y) 
d{x,y) d{r,s) / d{r,s) 


(14.42) 


Note that eq. (14.41), combined with eq. (14.31), gives us a useful identity for 
partial derivatives. 



y 



(14.43) 


An important application of eq. (14.43) can occur when you are looking for a 
Maxwell relation for a derivative like the following. 



T,N 


(14.44) 


If you try to find a transform of the fundamental relation, dU = TdS — Pdv + ydN, 
you run into trouble because S is in the denominator of eq. (14.44) and T is being 
held constant. However, we use eq. (14.41) to take the reciprocal of eq. (14.44), for 
which a Maxwell relation can be found. 


1 



\dV) t n ) v,N 


(14.45) 


14.7 Working with Jacobians 

As a first example of how to use Jacobians to derive thermodynamic identities, consider 
the partial derivative of pressure with respect to temperature, holding V and N 
constant. 



V,N 


(14.46) 


To simplify the notation, we will suppress the explicit subscript N. 

The first step is always to take the partial derivative you wish to simplify, and 
express it as a Jacobian using eq. (14.31). 


d{P,V) 

d{T,V) 


dP 

Irr 


V 


(14.47) 
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The next step is usually to insert d{P, T) into the Jacobian, using either eq. (14.34) 
or eq. (14.42). The reason is that the second derivatives given in Section 14.4—which 
constitute the standard set—are all derivatives with respect to P, holding T constant, 
or with respect to T, holding P constant. Inserting d{P,T) is a step in the right 
direction in either case. 


d{P,V) ^ d{P,V) / d{T,V) 
d (T, V) d (P, T) / d (P, T) 


(14.48) 


Next, we have to line up the variables in the Jacobians to produce the correct signs, 
remembering to change the sign of the Jacobian every time we exchange variables. 


d{P,V) d{P,V) /d{V,T) d{V,P) /d{V,T) 
d (P, V) ~ ~d{P,T) / d (P, T) “ ~ d{T,P) / d (P, T) 


(14.49) 


The minus sign came from switching T and V in the second factor. I also switched 
both top and bottom in the first factor to bring it into the same order as the form 
given above. This is not really necessary, but it might be helpful as a memory aid. 
Next, we can transform back to partial derivatives. 



V 



(14.50) 


Using the standard expressions for the coefficient of thermal expansion 


a = 


1 

V 



and the isothermal compressibility 


kt = — 


1 

V 



(14.51) 


(14.52) 


eq. (14.50) becomes 


To see why eq. (14.53) might be useful, consider the properties of lead, which has 
a fairly small coefficient of expansion. 

a(Pb) = 8.4 X lO"*^ K-^ (14.54) 

Lead also has a small isothermal compressibility. 


KT{Pb) = 2.44 X 


( 14 . 55 ) 
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Inserting these values into eq. (14.53), we find the change in pressure with temperature 
for a constant volume for lead. 



(14.56) 


As might be expected, it is very difficult to maintain lead at a constant volume as 
the temperature increases. A direct experiment to measure the change in pressure at 
constant volume would be extremely difficult. However, since a and kt are known, we 
can obtain the result in eq. (14.56) quite easily. 

This is a relatively simple case. The most common additional complexity is that 
you might have to use a Maxwell relation after you have transformed the derivatives, 
but it must be confessed that some identities can be quite challenging to derive. 

14.8 Examples of Identity Derivations 

The following two examples illustrate further techniques for proving thermodynamic 
identities using Jacobians. They are of interest both for the methods used and for the 
identities themselves. 


14.8.1 The Joule Thomson Effect 

In a ‘throttling’ procedure, gas is continuously forced through a porous plug with a 
high-pressure Pa on one side and a low-pressure Pb on the other. The initial and final 
states are in equilibrium. If we consider a volume Va of gas on the left, it will take 
up a volume Vb after it passes to the right. The energy of this amount of gas on the 
right is determined by conservation of energy. 


(14.57) 


Ub = Ua + PaVa - PbVb 


Rearranging this equation, we find that the enthalpy is unchanged by the process, 
even though the process is clearly irreversible. 


(14.58) 


Ha = Ua + PaVa = Ub + PbVb = Hb 


If the pressure change is small, the temperature change is given by the partial 
derivative. 



(14.59) 
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The partial derivative in eq. (14.59) is called the Joule-Thomson coefficient, fLjx-^ 


E-JT 



(14.60) 


We can express the Joule-Thomson coefficient in terms of the standard set of deriva¬ 
tives using Jacobians. 


(dT\ d{T,H) 


d{T,H) / d{P,H) 
d{P,T) / d{P,T) 



(14.61) 


The derivative with respect to P can be evaluated by using the differential form of 
the fundamental relation in the enthalpy representation. (The term ptdN is omitted 
because the total number of particles is fixed.) 


dH = 



TdS+VdP 




dP 


-T 


T,N 

dV\ 
df) 


dP 


T.N 


V 


P,N 


-TVa+V 
-V(Ta-l) 


(14.62) 

(14.63) 


The proof of the Maxwell relation used in this derivation will be left as an exercise. 
We can also transform the other partial derivative in eq. (14.61) using eq. (14.62). 


dH 

Irf 


P,N 


= T('^ 
dT 

= Ncp 


P,N 


' dT 


P,N 


(14.64) 


Putting eqs. (14.61), (14.63), and (14.64) together, we find our final expression for the 
Joule-Thomson coefficient. 


^James Prescott Joule (1818-1889) was a British physicist who had worked on the free expansion of 
gases, and William Thomson (1824-1907) was an Irish physicist who continued along the lines of Joule’s 
investigations and discovered the Joule-Thomson effect in 1852. Thomson was later elevated to the peerage 
with the title Baron Kelvin, and the Joule-Thomson effect is therefore sometimes called the Joulc-Kelvin 
effect. 





150 Thermodynamic Identities 


i^JT = ^iTa- 1) (14.65) 

The Joule-Thomson effect is central to most refrigeration and air-conditioning 
systems. When the Joule-Thomson coefficient is positive, the drop in pressure across 
the porous plug produces a corresponding drop in temperature, for which we are often 
grateful. 


14.8.2 cp and Cv 

The thermodynamic identity linking cp and cy has already been alluded to in 
Section 14.4. It is central to the reduction to the three standard partial derivatives by 
eliminating cy- This derivation is one of the relatively few times that an expansion of 
the Jacobian turns out to be useful in proving thermodynamic identities. 

Begin the derivation with cy, suppressing the subscript N for simplicity. 


T (dS\ 

T d{S, V) 

~ N d{T, V) 

_ T d{S,V) /d{T,V) 
~ Nd{T,P) / d{T,P) 


(14.66) 


The denominator of the last expression can be recognized as the compressibility. 


d{T,V) d{V,T) 

Wl^) ~ WTt) ~ \dP) T 


(14.67) 


The Jacobian in the numerator of the last expression in eq. (14.66) can be expressed 
by expanding the defining determinant. 


5(5',1^) fdS\ fdV\ fdS\ f dV\ 


(14.68) 


Now we can identify each of the four partial derivatives on the right 


of eq. (14.68). 



(14.69) 

(14.70) 

(14.71) 


p 


(14.72) 
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In eq. (14.71), a Maxwell relation has been used to relate the partial derivative to a. 
Putting the last six equations together, we obtain 


Cp = cv + 


a^TV 

Nkt 


(14.73) 


which is the required identity. 

Although it should not be necessary to memorize eq. (14.73), you should be able 
to derive it. 

14.9 General Strategy 

This section summarizes a useful strategy to reduce a given partial derivative to an 
algebraic expression containing only the three partial derivatives given in Section 14.4. 
These steps might be needed for attacking the most difficult derivations, but not all 
steps are necessary in most cases. 

1. Express the partial derivative as a Jacobian. 

2. If there are any thermodynamic potentials in the partial derivative, bring them 
to the numerator by applying eq. (14.42). Unless you have a good reason to do 
something else, insert d{T,P) in this step. 

3. Eliminate thermodynamic potentials if you know the derivative. For example: 



4. Eliminate thermodynamic potentials by using the differential form of the funda¬ 
mental relation. For example: if you want to evaluate 



use 

dF = -SdT - PdV + fidN 

to find 

\dp)^,,- KdPJr^N 

Note that the first and last terms on the right in this example vanish. 

5. If the system is extensive, bring /i to the numerator and eliminate it using the 
Gibbs-Duhem relation, eq. (13.9). 
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For example: 



6. Move the entropy to the numerator using Jacobians and eliminate it by either 
identifying the partial derivative of the entropy as a specific heat if the derivative 
is with respect to T, or using a Maxwell relation if the derivative is with respect 
to pressure. 

7. Bring V into the numerator and eliminate the partial derivative in favor of a or 

8. Eliminate cy in favor of cp, using the identity in eq. (14.73), derived in the 
previous section. (This last step is not always needed, since cy is sometimes 
easier to measure than cp.) 

14.10 Problems 


Problem 14.1 
Maxwell relations 

Note: This assignment is useless if you just look up the answers in a textbook. You 
will not have a book during examinations, so you should do this assignment with book 
and notes closed. 

Transform the following partial derivatives using Maxwell relations. 

1 . _fd^\ 

2 . 

3. 

4. 

5. 

6 . 

7 . 
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Problem 14.2 

A thermodynamic identity 

Express the following thermodynamic derivative in terms of a^KT,cv, and cp using 
Jacobians. 




S,N 


= 7 


Problem 14.3 

Prove the following thermodynamic identity 

fdcv\ _ T rd^P\ 
\dV N[dT^)y^ 


Problem 14.4 

Express the following partial derivative in terms of the usual standard 
quantities 



T,N 


Problem 14.5 

Yet another thermodynamic identity 

Prove the following thermodynamic derivative and determine what should replace the 
question marks. Note that you do not have to reduce this to the standard expressions, 
but can leave it in terms of partial derivatives 

N \dT ) W?/s,Af 


Problem 14.6 
Compressibility identity 

In analogy to the isothermal compressibility, 

1 /dV\ 
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we can define the adiabatic compressibility. 


ns = -y 


1 



S,N 


The name is due to the fact that TdS = ctQ for quasi-static processes. 

In analogy to the derivation for the difference in the specific heats at constant 
pressure and constant temperature, derive the following thermodynamic identity: 


TVa^ 

Ncp 


KS = kt - 


Problem 14.7 

Callen’s horrible example of a partial derivative 

In his classic book on thermodynamics, Herbert Callen used the following derivative 
as an example. 



However, he did not complete the derivation. Your task is to reduce this partial 
derivative to contain only the standard partial derivatives a^Kp^cp, and cy, and, 
of course, any of the first derivatives, T, S', F, V, or N. 

This derivation will require all the techniques that you have learned for transform¬ 
ing partial derivatives. Have fun! 

Problem 14.8 
A useful identity 

Prove the identity: 


d{V,S) NVcvkt 
d (T, P) ~ T 


Problem 14.9 
The TdS equations 

Prove the following three TdS equations. [N is held constant throughout.] 


1. First TdS equation 


TdS = NcvdT+{Ta/KT)dV 


2. Second TdS equation 
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Prove: 


TdS = NcpdT - TVadP 


3. Third TdS equation 
Three forms: 


or 


TdS = Ncp 



dV + Ncv 

p 



dP 


TdS = ^^dV + Ncv 
Va 



dP 


or 


TdS = 


^dv+^^^^dP 

Ya a 


Problem 14.10 
Another useful identity 

Since we will be using it in statistical mechanics, prove the following identity. 


V d/3 


= U 

V,N 


1 

k^- 


where j3 
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Extremum Principles 


/ worked my way up from nothing to a state of extreme poverty. 

Groucho Marx 


In Part I we defined the entropy, derived a formal expression for it in classical 
statistical mechanics, and established that it is a maximum in equilibrium for an 
isolated composite system with constant energy. This last result is important for many 
reasons, including providing us with a systematic way of calculating the equilibrium 
values of quantities when a constraint is released. 

There are several alternative principles, called extremum principles, that also allow 
us to calculate equilibrium properties under different thermodynamic conditions. The 
derivation of these principles is the main purpose of this chapter. 


In this chapter, we derive extremum principles with respect to internal variables in 
a composite system. For example, energy could be exchanged through a diathermal 
wall, and the extremum condition would tell us how much energy was in each 
subsystem in equilibrium. 

All extremum principles in this chapter are with respect to extensive variables; 
that is, variables that describe how much of something is in a subsystem: for a 
simple system, these are just the values of [/„, Va, or Na for subsystem a. 

This chapter does not contain any extremum principles with respect to the 
intensive variables, such as T, P, or p. 

In Chapter 16 we will examine thermodynamic stability between subsystems, 
which will lead to conditions on second derivatives with respect to the total volume 
or number of particles of the subsystems. 


To begin with, corresponding to the principle that the entropy is maximized when 
the energy is constant, there is an ‘energy minimum’ principle, which states that the 
energy is minimized when the entropy is held constant. 

15.1 Energy Minimum Principle 

To derive the energy minimum principle, start with the entropy maximum principle. 
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Consider the entropy, S, of a composite system as a function of its energy, U, 
and some other parameter, X, which describes the distribution of the amount of 
something in the composite system. For example, X could be the number of particles 
in a subsystem, Na- When a hole is made in the wall between subsystems A and B, 
the value of X = Na at equilibrium would maximize the entropy. In other examples 
we have looked at, X could be the energy of a subsystem or the volume of a subsystem 
that might vary with the position of a piston. It could not be the temperature, pressure, 
or chemical potential of a subsystem, because they are not extensive; that is, they do 
not represent the amount of something. 

In equilibrium, the entropy is at a maximum with respect to variations in X, 
holding U constant. This means that the first partial derivative must vanish, and the 
second partial derivative must be negative. 



= 0 

(15.1) 

< 0 

(15.2) 


Near equilibrium, a small change in the entropy can be represented to leading order 
by changes in U and X. 


Since 


dS 


1 

2 V5X2 ) 


(dX)^ + 

u 





X 



Eq. (15.3) can also be written as 


dS 


1 fd^S\ 

2 


(dX)2 + 
u 



(15.3) 


(15.4) 


(15.5) 


Now comes the subtle part. 

For the analysis of the entropy maximum principle, we isolated a composite system 
and released an internal constraint. Since the composite system was isolated, its 
total energy remained constant. The composite system went to the most probable 
macroscopic state after release of the internal constraint, and the total entropy went 
to its maximum. Because of the increase in entropy, the process was irreversible. 

Now we are considering a quasi-static process without heat exchange with the rest 
of the universe, so that the entropy of the composite system is constant. However, for 
the process to be quasi-static we cannot simply release the constraint, as this would 
initiate an irreversible process, and the total entropy would increase. Outside forces 
are required to change the constraints slowly to maintain equilibrium conditions. This 
means that the energy of the composite system for this process is not constant. 
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<=> 













Fig. 15.1 Piston experiments. The two diagrams each represent a cylinder that is sealed at both 
ends and is divided into two subvolumes by a moveable piston. Both systems are thermally isolated, 
and cannot exchange heat energy with the rest of the universe. When the piston in the upper 
cylinder is released it will move freely until it comes to an equilibrium position that maximizes the 
entropy. In the lower cylinder, the piston is attached to a rod that can be pushed or pulled from 
outside the cylinder. The piston is moved to a position at which there is no net force on the piston 
from the gas in the two subvolumes of the cylinder. 


The distinction between the entropy maximum and energy minimum principles 
can be illustrated by considering two experiments with a thermally isolated cylinder 
containing a piston that separates two volumes of gas. The two experiments are 
illustrated in Fig. 15.1. 

Entropy maximum experiment: Here the piston is simply released, as illustrated 
in the upper picture in 15.1. The total energy is conserved, so that entropy is 
maximized at constant energy. 

Energy minimum experiment: Here the piston is connected by a rod to some¬ 
thing outside the cylinder, as illustrated in the lower picture in 15.1. The piston 
is moved quasi-statically to a position at which the net force due to the pressure 
from the gas in the two subvolumes is zero. The total energy of the cylinder has 
been reduced. However, since the process is quasi-static and there has been no 
heat exchange between the cylinder and the rest of the universe, the total entropy 
of the system is unchanged. Energy is minimized at constant entropy. 


To determine the change in energy for the process in which the piston is held and 
moved quasi-statically to the equilibrium positon, we can turn eq. (15.5) around to 
find the leading behavior of dU as a function of dS and dX. 


d^s\ 

9X2 ) 


{dX)^ + TdS 


dU^-T 


u 


(15.6) 
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Eq. (15.6) gives us a new way of expressing the equilibrium conditions: 



(15.7) 


and 



(15.8) 


Since the first partial derivative vanishes, and the second partial derivative is 
positive, the energy is a minimum at equilibrium for constant entropy. This is 
another consequence of the Second Law of Thermodynamics. Indeed, it is equivalent 
to the maximization of the entropy in an isolated system as an expression of the 
Second Law. 

A consequence of eq. (15.8) is that the maximum work that you can extract from 
a system in a process with fixed entropy is given by the change in energy of the 
system. To see this, note that maximum efficiency is always achieved by quasi-static 
processes. The differential form of the fundamental relation is valid for quasi-static 
processes. 


(15.9) 


dU = TdS - PdV - ndN = TdS + dW - ptdN 


We can see that if the entropy and the number of particles are held constant, the 
change in the energy of the system is equal to the work done on it. 


(15.10) 


dU = dW 


This equation is actually a bit more general than the derivation might indicate. We 
might imagine a composite system doing work without changing its total volume, as 
in the example at the bottom of Fig. 15.1. The system could do work by exerting a 
force on the rod, due to a pressure difference between the two subsystems. 

15.2 Minimum Principle for the Helmholtz Free Energy 

Both the entropy maximum principle and the energy minimum principle apply to 
a situation in which the composite system is thermally insulated from the rest of 
the universe. Now consider a different situation, in which the composite system 
(and its constituent subsystems) are in contact with a thermal reservoir, so that 
the temperature of the composite system is held constant. The entropy maximum 
and energy minimum principles no longer apply, because we can transfer energy and 
entropy in and out of the thermal reservoir. Nevertheless, we can find an extremum 
principle under these conditions. 

Under constant temperature conditions, it is natural to use the Helmholtz free 
energy, which is the Legendre transform of U with respect to T. 


F = U[T] = U-TS 


( 15 . 11 ) 
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To analyze the new situation at constant temperature, consider the explicit case 
of the composite system of interest in contact with a thermal reservoir at temperature 
Tfi, but thermally isolated from the rest of the universe. At equilibrium with respect 
to some thermodynamic variable X (subsystem volume, number of particles, and so 
on), the total energy is a minimum. This gives us the equations 

^{U+Un) = 0 ( 15 . 12 ) 

and 

-^{U+Uh)>0 ( 15 . 13 ) 

subject to the condition that the entropy is constant (called the ‘isentropic’ condition). 


_d_ 


{S + Sr) = Q 


( 15 . 14 ) 


The only energy exchange between the reservoir and the system is in the form of 
heat. The differential form of the fundamental relation for the reservoir simplifies to 
dUfi = TrcISr, which gives us an equation for the partial derivatives. 


OUr _ dSR 
dX ^ dX 


( 15 . 15 ) 


The partial derivative of the Helmholtz free energy of the system can now be 
transformed to show that it vanishes in equilibrium. 


dF 


JL 

A 

dU 

M 

dU 

dU 

dX 

JL 

dX 

0 


{U - TS) 


{U - TrS) 

-Tr^ 

^dX 


+ Tr 


OSr 


dX 

dUR 

H- - 

dX 

{U + Ur) 


( 15 . 16 ) 


A similar derivation shows that A is a minimum at equilibrium. 
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d^F 


d^U d^S 

dX'^ ^ dX'^ 
d^U O^Ur 
dX^ ^ 5X2 


> 0 


(15.17) 


The maximum work that can be obtained from a system in contact with a thermal 
reservoir is not given by the change in U for the system, because energy can be 
extracted from the reservoir. However, the maximum work is given by the change in 
Helmholtz free energy. 

As in the previous section, we note that maximum efficiency is always obtained 
with quasi-static processes. The differential form of the fundamental relation in the 
Helmholtz free energy representation is valid for quasi-static processes at constant 
temperature. 


dF = -SdT - PdV - ndN = -SdT + dW - fidN (15.18) 

We can see that if the temperature and the number of particles are held constant, the 
change in the Helmholtz free energy of the system is equal to the work done on it. 

dF = dW (15.19) 

As was the case for the corresponding eq. (15.10), the work referred to in eq. (15.19) 
is more general than —PdV for a composite system. 


As was the case for the energy minimum principle, the minimum principle for the 
Helmholtz free energy refers to processes in which a constraint on the amount of 
something in a subsystem is released quasi-statically. It is therefore not valid to 
substitute the temperature or pressure of a subsystem for X in eq. (15.17), even 
though it might be tempting. We will discuss second derivatives with respect to 
temperature, pressure, and chemical potential in Chapter 16. 
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15.3 Minimum Principle for the Enthalpy 

When the pressure is held constant, minimizing the enthalpy gives us the condition of 
equilibrium. The proof is similar to those in the previous sections. 

Let the system of interest be in contact with a constant pressure reservoir at 
pressure Pr. At equilibrium with respect to some thermodynamic variable X, the 
total energy is again a minimum, subject to the constant total volume condition. 

^{V+Vr) = 0 ( 15 . 20 ) 


The only energy exchange between the reservoir and the system is in the form of 
work. The differential form of the fundamental relation for the reservoir simplifies to 
dUR = —PRdVR, which gives us an equation for the partial derivatives. 


QUr _ OVr 
dX ~ ^ dX 


( 15 . 21 ) 


The partial derivative of the enthalpy of the system can now be transformed to 
show that it vanishes in equilibrium. 


dH 




^WPP.V) 

dX ^dX 
dU dVR 
dX ^ dX 


dU 

d 


BUr 

dX 




A similar derivation shows that is a minimum at equilibrium. 

n2 u ^2 

Wx ^ Wx + 

d^u „ d^v 
dX^ ^dX2 
d^U B^Vr 
3X^ ^ BX^ 


( 15 . 22 ) 


( 15 . 23 ) 
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_ d'^U d^UR 
~ dX^ ^ 5X2 

> 0 

Changes in the enthalpy can be related to the heat added to a system. For reversible 
processes holding both P and N constant, TdS = ctQ. 


dH = TdS + VdP + fidN 

(15.24) 

dH = dQ 

(15.25) 


Because of eq. (15.25), the enthalpy is often referred to as the ‘heat content’ of a 
system. Although this terminology is common in chemistry, I do not recommend it. 
It can be quite confusing, since you can add arbitrarily large amounts of heat to a 
system without altering its state, as long as you also remove energy by having the 
system do work. Heat is not a state function and dQ is not an exact differential. 
Eq. (15.25) equates an exact and an inexact differential only with the restrictions 
of dN = 0 and dP = 0, which makes variations in the system one-dimensional. 


15.4 Minimum Principle for the Gibbs Free Energy 


If a system is in contact with both a thermal reservoir and a pressure reservoir, the 
Gibbs free energy is minimized at equilibrium. 


dG 

d^G 

5X2 


= 0 


> 0 


(15.26) 

(15.27) 


The proof of these equations is left as an exercise. 

For a composite system in contact with both a thermal reservoir and a pressure 
reservoir, the Gibbs free energy also allows us to calculate the maximum work that 
can be obtained by a reversible process. If a composite system can do work on the 
outside world through a rod, as in the bottom diagram in Fig. 15.1, the differential 
form of the fundamental relation will be modified by the addition of a term dWx ■ 


dG = -SdT + VdP + d:Wx + fLdN (15.28) 

For a reversible process with no leaks {dN = 0), constant pressure {dP = 0), and 
constant temperature {dT = 0), the change in the Gibbs free energy is equal to the 
work done on the system. 
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dG = dWx 


(15.29) 


15.5 Exergy 

In engineering applications it is common to introduce another thermodynamic poten¬ 
tial called the ‘exergy’, E, which is of considerable practical value. ^ Exergy is used 
in the common engineering situation in which the environment provides a reversible 
reservoir at temperature, To, and pressure, Pq, which are usually taken to be the 
atmospheric temperature and pressure. 

In engineering textbooks, the exergy is often denoted by the letter E, which could 
lead to confusion with the energy. This is especially true for books in which the 
energy is denoted by E, making the distinction rely on a difference in font. Reader 
beware! 


The exergy is defined as the amount of useful work that can be extracted from the 
system and its environment. If we use X to denote the internal variable (or variables) 
that change when work is done by the system, we can write the exergy as 

E{To,Po,X) = -W{X) (15.30) 

where W is the work done on the system. (Note that engineering textbooks often use 
a different sign convention for W.) 

Clearly, exergy is used in essentially the same situation described in the previous 
section on the Gibbs free energy, so it is not surprising that the exergy and the Gibbs 
free energy are closely related. The main difference is that the exergy is defined so 
that it is zero in the ‘dead state’, in which G takes on its minimum value and X takes 
on the corresponding value Xq- The dead state is characterized by the impossibility 
of extracting any more useful work. If we subtract the value taken on by the Gibbs 
free energy in the dead state, we can make explicit the connection to the exergy. 

E{To, Po, X) = G{To, Po, X) - G(To, P„, Xo) (15.31) 

The equation for the exergy in engineering texts is usually written without reference 
to the Gibbs free energy. 

E(r„, Po, X) = {u- Uo) +Po{V- Vo) -To{S- So) +KE + PE (15.32) 

Note that the kinetic energy, KE, and the potential energy, PE, are included explicitly 
in this equation, while they are implicit in eq. (15.31). 


^The concept of exergy was first introduced in the nineteenth century by the German chemist Friedrich 
Wilhelm Ostwald (1853-1932), who in 1909 was awarded the Nobel Prize in Chemistry. Ostwald divided 
energy into ‘Exergie’(exergy = useful energy) and ‘Anergie’ (anergy = wasted energy). 
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It is a common engineering convention that although the kinetic and potential 
energy are included in the exergy, they are not included in the internal energy of 
the system, as we have done in the rest of this book. Note that the zero of potential 
energy is also assumed to be determined by the environment. 


Since the exergy is the maximum amount of work that can be obtained from a 
system exposed to the atmospheric temperature and pressure, it is an extremely useful 
concept in engineering. An engineer is concerned with how much work can be obtained 
from a machine in a real environment, and this is exactly what the exergy provides. 

15.6 Maximum Principle for Massieu Functions 

Just as minimization principles can be found for F, H, and G, which are Legendre 
transforms of U, maximization principles can be found for the Massieu functions, 
which are Legendre transforms of the entropy. 

The maximization principles can be seen most quickly from the relations 


S'[l/r] = -F/T 

(15.33) 

S[\/T,P/T] = -G/T 

(15.34) 


and a comparison with the maximization principles for F and G. 

15.7 Summary 

The extremum principles derived in this chapter are central to the structure of 
thermodynamics for several reasons. 

1. We will use them immediately in Chapter 16 to derive stability conditions for 
general thermodynamic systems. 

2. When stability conditions are violated, the resulting instability leads to a phase 
transition, which is the topic of Chapter 17. 

3. Finally, in Parts III and IV, we will often find that the most direct calculations 
in statistical mechanics lead to thermodynamic potentials other than the energy 
or the entropy. The extremum principles in the current chapter are essential for 
finding equilibrium conditions directly from such statistical mechanics results. 


15.8 Problems 


Problem 15.1 
Extremum conditions 


1. Helmholtz free energy for the ideal gas 
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From the expression for the Helmholtz free energy of the ideal gas, calculate 
AF for an isothermal expansion from volume Va to Vb when the system is in 
equilibrium with a thermal reservoir at temperature Tfj. Compare this to the 
work done on the system during the same expansion. 

2. Gibbs free energy minimum principle 

Consider a composite system with a property X that can be varied externally. 
[For example, X could be the volume of a subsystem, as discussed in class.] 
Prove that at equilibrium. 



and 



3. Let dW be the work done on the system by manipulating the property X from 
outside the system. 

Prove that dG = dW, when the system is in contact with a reservoir that fixes 
the temperature at Tr and the pressure at Pr. 



16 

Stability Conditions 


No one welcomes chaos, but why crave stability and predictability? 
Hugh Mackay, Australian psychologist 


So far, we have implicitly assumed that the systems we have studied are stable. For 
example, we have been assuming that the density of a gas will remain uniform, rather 
than having most of the particles clump together in one part of the container, leaving 
the rest of the volume nearly empty. Gases are usually well behaved in this respect, 
but we all know from experience that molecules of H 2 O can clump together, form 
drops, and rain on us. 

The question of stability therefore leads naturally to a consideration of phase 
transitions, in which the properties of a material change drastically in response to 
small changes in temperature or pressure. 

In this chapter we will discuss the stability of thermodynamic systems. We will find 
that certain inequalities must be satisfied for a system to be stable. The violation of 
these inequalities signals a phase transition and a number of interesting and important 
phenomena—including rain. 

The methods used in this chapter are based on the extremum principles derived 
in Chapter 15. Those extremum principles were valid for composite systems, in which 
the energy, volume, or particle number of the subsystems was varied. 

In this chapter we look inside a composite system to see what conditions a 
subsystem must satisfy as a consequence of the extremum principles in Chapter 15. 
These are called the ‘intrinsic’ stability conditions. 

Since any thermodynamic system could become a subsystem of some composite 
system, the stability conditions we derive will be valid for all thermodynamic systems. 


16.1 Intrinsic Stability 

Several approaches have been developed during the history of thermodynamics to 
derive the intrinsic stability conditions. The one used here has the advantage of being 
mathematically simple. Its only drawback is that it might seem to be a special case, 
but it really does produce the most general intrinsic stability conditions. 

Although all stability conditions can be obtained from any of the extremum prin¬ 
ciples derived in Chapter 15, there are advantages to using different thermodynamic 
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potentials for different stability conditions to simplify the derivations, as we will see 
in the following sections. 

16.2 Stability Criteria based on tbe Energy Minimum Principle 

Consider two arbitrary thermodynamic systems. Denote the properties of the first as 
S, U, V, N, and so on, and distinguish the properties of the second with a tilde, S, 
U, V, N, and so forth. Combine the two systems to form a composite system, which 
is thermally isolated from the rest of the universe. 

Allow the subsystems to interact with each other through a partition, which can 
be fixed or moveable, diathermal or adiabatic, impervious to particles or not, as you 
choose. Denote the quantity being exchanged as X. As in the previous chapter, X can 
denote any variable that describes how much of something is contained in the system. 
For example, the wall might be diathermal and X the energy of a subsystem, or the 
wall might be moveable and X the volume of a subsystem. However, X must be an 
extensive variable; it cannot be the temperature or pressure. 

16.2.1 Stability with Respect to Volume Changes 

As we saw in Section 15.1, the total energy is a minimum at equilibrium for constant 
total volume. Suppose we have a moveable wall (piston) separating the subsystems, 
as in the lower diagram in Fig. 15.1, and we use it to increase V to V + AV, while 
decreasing V to V — XV. The energy minimum principle would then demand that the 
change in total energy must be non-negative. 


XUtotai = U{s, V + AV, N) - U{S, V, N) 

+U{S, V - A V, JV) - U(S, V, N) 
> 0 


(16.1) 


The equality will hold only when AV = 0. The volume transferred need not be small; 
the inequality is completely general. 

Now take the special case that the properties of the two systems are identical. 
Eq. (16.1) can then be simplified to give an equation that must be true for a single 
system. 


U{S, V + AV, N) + U{S, V - AV, N) - 2U{S, V,N)>0 


(16.2) 


This equation must also hold for arbitrary values of AS'; it is not limited to small 
changes. 

Now divide both sides of eq. (16.2) by (AV)^ and take the limit of AV ^ 0. 


lim 


U{S,V + AV,N) + U{ 





(16.3) 


This gives us a stability condition on a second derivative. 
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Since we know that 



(16.4) 


Eq. (16.3) can also be expressed as a stability condition on a first derivative. 



> 0 


(16.5) 


If we define an isentropic (constant entropy) compressibility in analogy to the isother¬ 
mal compressibility in eq. (14.14), 


Ks = - 


1 

V 



we find that ks must also be positive for stability. 



-1 V 

\9PJs, 


(16.6) 


(16.7) 


or, since V > Q 


KS > 0 (16.8) 

This means that if you increase the pressure on a system, its volume will decrease. 
Eq. (16.8) is true for all thermodynamic systems. 

The stability conditions in this chapter are expressed as inequalities for second 
partial derivatives of the thermodynamic potentials. This makes them look very 
much like the extremum principles in Chapter 15. However, these inequalities do 
not represent extremum principles, because the corresponding first derivatives are 
not necessarily zero, as they must be at an extremum. 


16.2.2 Stability with Respect to Heat Transfer 

We can use the same method as in Subsection 16.2.1 to consider the consequences of 
transferring some amount of entropy between the subsystems. The energy minimum 
principle again demands that the change in total energy be non-negative. 

^Utotai = U{s + A5, E, N) - U{S, E, N) 

+U{S - AS, V, N) - U{S, V, N) 

> 0 


(16.9) 
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The equality will hold only when AS = 0. The entropy transferred need not be small; 
this inequality is also completely general. 

Now again take the special case that the properties of the two systems are identical. 
Eq. (16.9) can then be simplified to give an equation that must be true for a single 
system. 


U{S + AS, V, N) + U(S - AS, V, N) - 2U(S, V,N) >0 


(16.10) 


This equation must also hold for arbitrary values of AS; it is not limited to small 
changes in entropy . 

Divide both sides of eq. (16.10) by (AS")^ and take the limit of AS' —*■ 0. 


lim 

AS^O 


U{S + AS, V, N) + U{S - AS, V, N) - 2U(S, V, N) 


_ /d^U\ 


V,N 


Since we know that 


/m 


= T 


V,N 


eq. (16.11) can be expressed as a stability condition on a first derivative. 

> 0 


(i) 


> 0 

(16.11) 

(16.12) 

(16.13) 


V,N 


This inequality can be rewritten in terms of the specific heat at constant volume. 

T 


dT 

1)S 


= 1 


V,N 


as; 

df 


V,N 


Ncv 


> 0 


(16.14) 


Since the temperature must always be positive, in order for the system to be stable, 

cy > 0 (16.15) 

This means that if heat is added to a system, its temperature will increase, which 
certainly agrees with experience. Eq. (16.15) is valid for all thermodynamic systems. 


16.3 Stability Criteria based on the Helmholtz Free Energy 
Minimum Principle 

Now we extend the stability criteria to cases in which the temperature of the composite 
system is held constant. This makes it natural to use the Helmholtz free energy 
minimization principle. 

Recall that we do not (yet) have an stability condition involving derivatives with 
respect to temperature, so we will only look at the case of moving a piston to vary 
the volume. 
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The experiment we consider is the same as that illustrated in the lower picture 
in Fig. 15.1, except that instead of being thermally isolated, the entire cylinder is in 
contact with a heat bath at temperature T. 

The Helmholtz free energy minimization principle now leads to the condition: 


F(r, V + AV, N) + F{T, V - AT, N) - 2F(T, V,N) >0 


(16.16) 


This equation must also hold for arbitrary values of AV\ it is not limited to small 
changes in the volumes of the subsystems. 

Dividing eq. (16.16) by (AH)^ and taking the limit of AV —*■ 0, we find a new 
stability condition. 


F{T, V + AV, N) + F{T, V - AV, N) - 2F{T, V, N) 


lim 

Ay^o 


Since we know that 


we can rewrite eq. (16.17) as 


(AH) = 


/OF 

Vw 


fd^F\ 


T,N 


= -P 


T,N 


dP 

w 


> 0 


T,N 


Recalling the definition of the isothermal compressibility in eq. (14.14), 

-1 fdV\ 


Kt = 


Eq. (16.19) becomes 


dP 

dV 


V \dPJ 


-1 


T,N 


T,N 




V Kx 


> 0 


> 0 

(16.17) 

(16.18) 

(16.19) 

(16.20) 

(16.21) 


T,N 


This tells us that the isothermal compressibility is also non-negative for all systems. 

kt > 0 (16.22) 


16.4 Stability Criteria based on the Enthalpy 
Minimization Principle 

Next we consider a consequence of the enthalpy minimization principle. Here, we are 
looking at a situation in which the total pressure on the system and on each of its 
subsystems is constant. We will consider an experiment in which heat is transferred 
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between the identical subsystems. The derivation should look familiar, since it follows 
the same pattern used in previous derivations. 

The enthalpy minimization principle gives us the condition: 


H{S + AS, P, N) + H{S - AS, P, N) - 2H{S, P,N)>0 (16.23) 


This equation must also hold for arbitrary values of AS'; it is not limited to small 
changes. 

Dividing eq. (16.23) by (AS)^ and taking the limit of AS ^ 0, we find yet another 
stability condition. 


lim 

AS^O 


'H{S + AS, P, N) + H{S - AS, P, N) - 2H{S, P, N)' 
(AS)2 


Since we know that 



> 0 

P,N 

(16.24) 


we can rewrite eq. (16.24) as 




(16.25) 


(16.26) 


Recalling the definition of the specific heat at constant pressure in eq. (14.15), 




(16.27) 


Eq. (16.26) becomes 



P,N 


1 _ T 

7as\ “ 

\dT) p jY 


(16.28) 


This tells us that the specific heat 


at constant pressure is always non-negative. 


cp > 0 


(16.29) 


16.5 Inequalities for Compressibilities and Specific Heats 

In Chapter 14 we derived a relationship between the specific heats at constant pressure 
and constant volume. According to eq. (14.73), 

a^TV 

A ttj' 


Cp = cv + 


(16.30) 









Other Stability Criteria 173 


Since all quantities in the second term on the right are non-negative, we immediately 
have another inequality. 


cp > cv (16.31) 

The equality would only occur when a = 0, which can actually happen! Liquid water 
has a = 0, and its maximum density at a temperature of 3.98° C and atmospheric 
pressure. Note that we have no stability condition on the coefficient of thermal 
expansion, a, even though we expect most things to expand when heated. This is as 
it should be, since there are several examples of materials that contract when heated, 
including water just above the freezing point. This does not affect the inequality in 
eq. (16.31), since only the square of a occurs in eq. (16.30). 

There is an equation and an inequality linking ks and kt that are similar to those 
linking Cp and cy. However, they are more fun to derive yourself, so they will be left 
as an exercise. 

16.6 Other Stability Criteria 

When we derived stability conditions for various second derivatives, we explicitly 
excluded derivatives with respect to T, P, or Addressing this omission turns out 
to be quite easy, but requires method different from that used in Chapters 15 and 16. 
One example will reveal the nature of the derivation. 

We will derive a stability condition for the second derivative of the Helmholtz 
free energy with respect to temperature. We know that the first derivative gives the 
negative of the entropy. 


=-S 

dTjy^ry 


(16.32) 


Take the partial derivative of eq. (16.32) with respect to temperature to find an 
expression for the quantity in which we are interested. 



V,N 



(16.33) 


The right-hand side of eq. (16.33) is, of course, equal to —Ncy/T, which must be 
negative. However, we are more concerned with relating it to the second derivative 
of the energy with respect to entropy, because that reveals a general property of 
thermodynamic potentials. 

The first derivative of the energy with respect to entropy gives the temperature. 


dU 


= T 


V,N 


(16.34) 
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Now take the second derivative of U with respect to S. 


d^U 


>0 


V,N 


\dS 


V,N 


(16.35) 


The last inequality in eq. (16.35) is just the stability condition in eq. (16.11) that we 
derived earlier in this chapter. 

Since we have the identity 



V,N 



(16.36) 


we find that the second partial derivative of F with respect to T must be negative. 

/d^F\ _ _ 

\dT^ ) V,N ) V,N 



< 0 (16.37) 


Eq. (16.37) shows that the second partial derivative of F with respect to T must 
have the opposite sign from the second partial derivative of U with respect to S. It 
can be seen from the derivation that quite generally the second partial derivative of 
the Legendre transform {F in this case) with respect to the new variable (T) has 
the opposite sign from the second partial derivative of the original function ([/) with 
respect to the old variable (S'). It is also a general rule that the two second derivatives 
are negative reciprocals of each other. 

Because of the simple relationship between the second derivatives of Legendre 
transforms, we can skip the explicit derivations of the remaining inequalities and 
simply summarize them in Table 16.1. 


I highly recommend memorizing the results shown in Table 16.1. Remembering 
them is easy because of the simplicity of their derivation, and they can save you 
from avoidable errors. It is astonishing how often these inequalities are violated in 
published data. Some prominent scientists have even made a hobby of collecting 
examples from the literature. Do not let them find any exhibits for their collections 
in your work! 
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Table 16.1 Summary of inequalities for second partial derivatives of thermodynamic potentials 
that are required for stability. 


U{S,V,N) 

F{T,V,N) 

H{S,P,N) 

G{T,P,N) 


/d‘^U\ 

d‘^F\ 

) 

\d^ ) 

}d^G\ 

\Jm) 


V,N 


V,N 


P,N 


P,N 


> 0 
< 0 
> 0 
< 0 


/d‘^U\ 

d^F\ 


dV^) 

fd'^H 


T,N 




\dP^ J 

d^G 




S,N 


ap2 j 


T,N 


> 0 
> 0 
< 0 
< 0 


16.7 Problems 


Problem 16.1 
Thermodynamic stability 


1. Starting with the stability conditions on the second derivatives of the Helmholtz 
free energy, prove that for the Gibbs free energy we have the general inequality: 


dP^) 


< 0 


T,N 


2. Use this inequality to prove that the isothermal compressibility is positive. 


Problem 16.2 
A rubber band 

The essential issues in the following questions concern whether the sign of some 
quantity is positive or negative. Getting the signs right is essential! 

Consider an ordinary rubber band. If the length of the rubber band is L, and 
we apply a tension r, a small change in length will change the energy of the rubber 
band by —rdL. Assuming the number of molecules in the rubber band is fixed, the 
differential form of the fundamental relation in the energy representation is 

dU = TdS + rdL 

Note that the sign of the pdL term has a positive sign, in contrast to the more common 
—PdV term. 
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1 . 


2 . 


Experimental question! 


Determine the sign of the quantity 



experimentally. 

s 


Obtain a rubber band—preferably a clean rubber band. 

Stretch the rubber band quickly and, using your forehead as a thermometer, 
determine whether is becomes hotter or colder. If the rubber band is very clean, 
you might try using your lips, which are more sensitive. 

Since you are only interested in the sign of the derivative, you do not 
really have to carry out the experiment under true adiabatic (constant entropy) 
conditions. It will be sufficient if you simply do it quickly. 

Now imagine that one end of the rubber band is attached to a hook in the 
ceiling, while the other end is attached to a weight. After the system is allowed 
to come to equilibrium with the weight hanging down, the rubber band is 
heated with a hair dryer. 

Using the result of your experiment with a rubber band and your knowledge 
of thermodynamic identities and stability conditions, predict whether the 
weight will rise or fall. 
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Life is pleasant. Death is peaceful. It’s the transition that’s troublesome. 
Isaac Asimov (1920-1992), biochemist, and author of both 
science fiction and non-fiction books. 


One of the most interesting branches of thermal physics is the study of phase 
transitions. While most of thermodynamics is concerned with the consequences of 
analyticity in relating different measurable quantities, phase transitions occur at points 
where the analyticity postulate (Section 9.6.5) is violated. 

Examples of phase transitions abound. Water can freeze, going from a liquid to a 
solid state when the temperature is lowered. Water can also boil, going from a liquid 
state to a gaseous state when the temperature is raised. Iodine can sublimate, going 
directly from a solid to a gaseous state. In fact, almost all materials can exist in 
different states, with abrupt transitions between them. 

Water boiling and freezing and iodine sublimating are examples of first-order 
transitions; that is, phase transitions in which the extensive variables change dis- 
continuously. 

An example of a second-order transition is given by the magnetization of iron, 
which goes to zero continuously at a ‘critical’ temperature of 1044 A'. The partial 
derivative of the magnetization with respect to temperature is not only discontinuous, 
but it diverges as the critical temperature is approached from below. 

The classification of phase transitions as first order or second order is a hold-over 
from an early classification scheme due to the Austrian physicist Paul Ehrenfest 
(1880—1933). Ehrenfest classified phase transitions at which the n-th partial 
derivative of the free energy was discontinuous as being n-th order. For first-order 
phase transitions this classification is very useful, but for higher-order transitions 
it is less so. Phase transitions have turned out to be much more complex and 
interesting than Ehrenfest thought. In normal usage his definition of first-order 
transitions is retained, but ‘second-order’ generally refers to any transition with 
some sort of non-analytic behavior, but continuous first partial derivatives of the free 
energy. 
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17.1 The van der Waals Fluid 

To illustrate some of the basic ideas about phase transitions we will use a model for 
a fluid of interacting particles that was invented by the Dutch physicist, Johannes 
Diderik van der Waals (1837-1923), who was awarded the Nobel Prize in 1910. 

17.2 Derivation of the van der Waals Equation 

There are several ways of deriving the van der Waals equations, but they all begin 
with making approximations for the changes in the behavior of an ideal gas when 
the effects of interactions on the properties are included. As discussed in Chapter 7, a 
typical interaction between two particles has the form shown in Fig. 7.1. For very short 
distances the interactions between molecules are generally repulsive, while they are 
attractive at longer distances. In the van der Waals model the effects of the attractive 
and repulsive parts of the interactions between particles are considered separately. 
Rather than calculating the properties of the van der Waals fluid directly from a 
specific interaction potential of the form shown in Fig. 7.1, we will follow the usual 
procedure of introducing a parameter a for the overall strength of the attractive part 
of the interaction, and another parameter b for the strength of the repulsive part. 

We will start from the Helmholtz free energy for the ideal gas. {X is the usual 
constant.) 


Fig 


-NksT 



+ — In iksT) + X 


(17.1) 


Since an interaction of the form shown in Fig. 7.1 implies that each particle is 
attracted to the particles in its neighborhood, we would expect the average energy of 
attraction to be proportional to the density of particles in the neighbor of any given 
particle, which we will approximate by the averge density, N/V. The total energy of 
attraction for N particles can then be written as —aN'^/V, where a > 0 is a constant. 

Assuming that the most important effect of the repulsive part of the interaction 
potential is a reduction in the available volume, we subtract a correction term 
proportional to the number of particles, so that V ^ V — bN. 

Naturally, the constants a and b will be different for different fluids. 

With these two changes in eq. (17.1), we And the Helmholtz free energy for the 
van der Waals fluid. 


Fvdw = —NkgT 


In 


/V-bN\ 

) 


^ln{kBT)+X 



(17.2) 


From eq. (17.2) we can use the usual partial derivatives with respect to T and V 
to And two equations of state. 


NkeT aN^ 


V -bN R2 


(17.3) 
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and 


U= -NkeT-a 
2 



(17.4) 


by the usual methods. 

Note that both van der Waals equations of state, eqs. (17.3) and (17.4), are 
unchanged (invariant) if U, V, and N are multiplied by an arbitrary constant A, 
demonstrating that the van der Waals gas is extensive. This is because surfaces and 
interfaces are completely neglected in this model, and the fluid is assumed to be 
homogeneous. 

17.3 Behavior of the van der Waals Fluid 

At very high temperatures, the gas expands, the density is small, and V is large in 
comparison with the correction term bN. For low density the correction term for the 
pressure is also small, so that the predictions of the van der Waals equation, eq. (17.3), 
are very close to those of the equation for an ideal gas. 

On the other hand, as the temperature is reduced, the deviations from ideal- 
gas behavior become pronounced. If the temperature is below a certain critical 
temperature, T^, the deviations from the ideal-gas law are not only large, but the 
behavior is qualitatively different. 

Below Tf,, a plot of P{V) vs. V looks qualitatively like Fig. 17.1, while above 
the slope is everywhere negative. The value of the critical temperature, Tc, can be 
found from the condition that P(V) has an inflection point at which both 



(17.5) 


P I 


H 



A 


V 


Fig. 17.1 Schematic P — V plot of an isotherm (eonstant temperature) for the van der Waals 
equation at a low temperature. The marked points are used to illustrate the calculation of the 
location of the phase transition. They are the same points as those marked in Fig. 17.2. 
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and 



= 0 


With a little algebra, eqs. (17.5) and (17.6) can be solved to give: 


(17.6) 


Vc = 3bN 

(17.7) 

P — ^ 

^ 2752 

(17.8) 

, 8a 

= 276 

(17.9) 


A very interesting feature of the van der Waals equation is that although it contains 
only two arbitrary constants, a and b, it predicts the three critical values, Vc, Pc, and 
Tc- This means that the three values cannot be independent. Indeed, they can be 
combined to make a non-trivial prediction for the ratio 


PcVc 

NkeTc 


= 0.375 


(17.10) 


The measured values of this ratio vary for different materials, but not as much as 
might be guessed from such a simple theory. For example, the values of PcVc/NksTc 
for helium, water, and mercury are, respectively, 0.327, 0.233, and 0.909. 

If we define the reduced values V = VjVc, P = P/Pc, and T = T/Tc, we can write 
the van der Waals equation in dimensionless form as 


or 


P = 


8T 

W-1 


3 

V2 


(17.11) 


(f + (SV -ij =8f 


(17.12) 


17.4 Instabilities 

The qualitative differences between the PV plot for the low-temperature van der 
Waals gas and for an ideal gas are dramatic. The PV plot for an ideal gas is simply 
an hyperbola {PV = NksT). The plot of a low-temperature isotherm for the van der 
Waals gas, shown in Fig. 17.1, has the striking feature that the slope is positve in the 
region between points F and D. 


> 0 


dP 

W 


T,N 


(17.13) 
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This is surprising, because we know from Chapter 16 (see eq. (16.19)) that the 
derivative in eq. (17.13) must be negative for stability. Therefore, the region of the 
van der Waals plot from points F to H in Fig. 17.1 must be unstable. 

We noted that the van der Waals equation was derived under the assumption that 
the system was homogeneous. The violation of the stability condition in the region 
between points F and D marks where this assumption breaks down. This part of the 
curve must represent unphysical states. 

More insight into the unusual features of the van der Waals equation can be seen 
if the axes of Fig. 17.1 are flipped, as shown in Fig. 17.2. 

From Fig. 17.2 it can be seen that the van der Waals equation actually predicts 
that the volume is a triple-valued function of the pressure in the region between points 
B and FI. The possibility of three solutions can also be seen directly from the form of 
the van der Waals equation. If eq. (17.3) is rewritten as 

{PV^ + aN^) (V - bN) = NksTV^ (17.14) 

and the left side of eq. (17.14) is expanded, the van der Waals equation is seen to 
be a cubic equation in V for fixed P. It can therefore have either one or three real 
solutions, which is consistent with Fig. 17.2. 

To And the stable states predicted by the van der Waals equation we must eliminate 
the section of the curve between points D and F on the basis of instability, but we 



Fig. 17.2 Schematic V — P plot of an isotherm (constant temperature) for the van der Waals 
equation at a low temperature. This is a plot of the same function as in Fig. 1 7. 1, but the axes 
have been flipped. The marked points are used to illustrate the ealculation of the location of the 
phase transition. They are the same points as those marked in Fig. 17.1. 
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still have to choose between the two remaining solutions. This will be done in the next 
section. 


17.5 The Liquid-Gas Phase Transition 

To resolve the problem of multiple solutions of the van der Waals equation, we need to 
go back to Chapter 16 and investigate the stability of the high- and low-density states 
in Fig. 17.2. For the stable state, the Gibbs free energy at constant temperature and 
pressure should be a minimum. Note that this criterion is not restricted to infinitesimal 
variations; it is also valid for the large differences in the volume between the states on 
the upper and lower branches in Fig. 17.2. 

To use the stability condition on the Gibbs free energy, G, we must be able to 
calculate it. Since this is an extensive system, we know two very helpful things. 

First, as shown in eq. (13.24) of Chapter 13, 

G =U -TS + PV = nN (17.15) 


so that the condition that G be a minimum is the same as the condition that /i be a 
minimum. 

Next, the Gibbs-Duhem equation, given in eq. (13.9), 


dfi = 



(17.16) 


is valid for the van der Waals fluid. Since Fig. 17.2 is an isotherm (constant temper¬ 
ature), we can set dT = 0 in eq. (17.16) and integrate it to find G or ^ as a function 
of P. 


G = nN 


VdP 


(17.17) 


Starting with point A in Fig. 17.2 and integrating eq. (17.17) to point P, we 
find the increasing part of the curve shown in Fig. 17.3. Note that the curvature 
of this part of the plot is negative because the integrand (V) is decreasing. As we 
continue to integrate along the curve in Fig. 17.2 from point D to point F, we are 
moving in the negative P-direction, so the contributions of the integral are negative, 
and the value of G = decreases. When we reach point F and continue to point 
7, we are again integrating in the positive P-direction and the value of G = 
increases. 

The integration of the van der Waals equation of state in Fig. 17.2, using eq. (17.17), 
is peculiar for at least two reasons. First, since the function V = V(P) is mul¬ 
tivalued, the integration must proceed in three stages: initially in the positive 
P-direction, then in the negative P-direction, and finally in the positive P-direction 
again. The more problematic feature is that we have already established that the 
states corresponding to the part of the function that lies between points 77 and F in 
Figs. 17.1 and 17.2 are unstable. They could only be regarded as equilibrium states if 
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they somehow were subject to an added constraint that they remain homogeneous. 
However, since we only need the van der Waals equation in the unstable region for 
the formal integral, and that part of the curve is an analytic continuation of the 
van der Waals isotherm, the procedure is valid—strange, but valid. 


The parts of the curve in Fig. 17.3 corresponding to equilibrium states are the 
two sections from point A to C and from point G to /, which have the minimum 
Gibbs free energy. Note that although the points C and G are distinct in Figs. 17.1 
and 17.2, they have the same Gibbs free energy and occupy the same location in 
Fig. 17.3. 

There are two kinds of instability that occur along the unstable parts of the curve. 
First, we have already noted that states corresponding to points between D and F 
are unstable to small perturbations because dP/dV > 0 in that region. On the other 
hand, dPjdV < 0 in the regions G to D and F to G, so that the corresponding states 
are stable to small perturbations. Nevertheless, they are still not equilibrium states, 
because there exist states with lower Gibbs free energy. They are unstable in a second 
sense: they are ‘globally’ unstable with respect to a large change. The term for such 
states is ‘metastable’. Because they are stable to small perturbations, metastable states 
can exist for a long time. However, eventually a large fluctuation will occur and the 
system will make a transition to the true equilibrium state, which will lower the Gibbs 
free energy. 



P 


Fig. 17.3 Schematic G — P plot for the van der Waals fluid at a low temperature. The function 
G = G{P) is obtained by performing the integral in eg. (17.17) on the van der Waals equation of 
state shown in Fig. 17.2. 
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Fig. 17.4 Schematic P — V plot of an isotherm (eonstant temperature) for the van der Waals 
equation at a low temperature. The labeled points eoincide with those in Fig. 17.1. By the 
Maxwell construction, the two shaded areas must have equal areas when the horizontal line 
bounding them is at the correct pressure to have equilibrium between the liquid and gas 
phases. 


17.6 Maxwell Construction 

The integration of the van der Waals equation of state in Fig. 17.2, using eq. (17.17), 
includes a positive contribution from point C to D, followed by a smaller negative 
contribution from D to E. The sum of these two contributions gives a net positive 
contribution to the Gibbs free energy. The magnitude of this net positive contribution 
is the shaded area under the curve EDC in Fig. 17.4. 

The integral from E to F gives a negative contribution, and it is followed by a 
smaller positive contribution from F to G, giving a net negative contribution. The 
magnitude of this contribution is the shaded area above the curve EFG in Fig. 17.4. 

Since the total integral from the point C to the point G must vanish (G and G 
have the same Gibbs free energy), the two shaded regions in Fig. 17.4 must have equal 
areas. The procedure of adjusting the position of the line GG until the two areas are 
equal is known as the Maxwell construction, first noted by the English physicist James 
Glerk Maxwell (1831-1879). 

17.7 Coexistent Phases 

The points G and G in Figs. 17.1, 17.2, 17.3, or 17.4 indicate distinct phases that 
are in equilibrium with each other at the same temperature and pressure. The fact 
that they are in equilibrium with each other is demonstrated by their having the same 
Gibbs free energy in Fig. 17.3. That they are distinct phases is clear because of the 
difference in their volumes. 

The phase corresponding to point G in the figures corresponds to a small volume 
and a high density. We will call it the liquid phase to distinguish it from the gas phase 
corresponding to point G, which has a large volume and small density. 
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Fig. 17.5 Schematic T — V plot of the coexistence diagram for the van der Waals fluid. The 
solid line indicates the liquid and gas phase volumes at each temperature. The dot at the top of 
the curve indicates the critical point. The dashed curves indicate the spinodals, which correspond 
to the points D and F in Figs. 17.1 through 17.f. Regions of liquid and gas phases are indicated, 
as well as regions of unstable and metastable states. 


For each temperature, we can draw an isotherm and perform the Maxwell construc¬ 
tion to determine the pressure at which the liquid and gas phases are in equilibrium 
with each other, as well as the volumes of the liquid and gas phases. 

Fig. 17.5 shows a qualitative plot of the relationship between the liquid and 
gas volumes at coexistence and the temperature. The area under the solid curve is 
known as the coexistence region, because those temperatures and volumes can only be 
achieved by having distinct liquid and gas phases coexisting next to each other. The 
dot at the top of the coexistence curve indicates the critical point, with the critical 
temperature Tc and the critical volume 14. As the temperature increases toward Tc 
from below, the volumes of the liquid and gas phases approach each other. Above Tc, 
there is no distinction between liquid and gas—only a single fluid phase. 

The dashed lines in Fig. 17.5 correspond to the points D and F in Figs. 17.1 through 
17.4. They are known as ‘spinodals’ and represent the boundaries of the metastable 
regions. All states under the dashed lines are unstable. 

17.8 Phase Diagram 

Equilibrium between liquid and gas phases occurs at a specific pressure for any given 
temperature, as determined by the Maxwell construction. The locus of the coexistence 
points on a T — T diagram is plotted schematically in Fig. 17.6. 

The curve in Fig. 17.6 is known as the ‘coexistence curve’ and separates regions of 
liquid and gas. At points along the curve, the liquid and gas phases coexist. The curve 
ends in a critical point with, logically enough, critical values of the temperature, Tc, 
and pressure. Pc- 

At temperatures or pressures above the critical point, there is no distinction 
between liquid and gas; there is only a fluid. This has the curious consequence that, 
at least in some sense, liquid and gas phases are different aspects of the same phase. 
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Fig. 17.6 Schematic P — T plot of the phase diagram of the van der Waals fluid. The solid line 
separates the liquid and gas phase regions. The dot at the end of the curve indicates the critical 
point. 

Consider a process that starts with a liquid on the coexistence curve at a temper¬ 
ature To and pressure Pq. Raise the temperature of the liquid at constant pressure. 
When the temperature is above Tc, the pressure is increased until it is above Pc- 
At this point, the temperature is again lowered until it is at the original value at 
Tq. Now lower the pressure until the original value of Pq is reached. The system has 
now returned to the original temperature and pressure, but on the other side of the 
coexistence curve. It is now in the gas phase. 

Since the coexistence curve was not crossed anywhere in the entire process, the 
fluid was taken smoothly from the liquid to the gas phase without undergoing a phase 
transition. The initial (liquid) and final (gas) phases must, in some sense, represent 
the same phase. Considering how obvious it seems that water and steam are different, 
it might be regarded as rather surprising to discover that they are not fundamentally 
different after all. 

17.9 Helmholtz Free Energy 

The graph of the Helmholtz free energy F as a function of the volume V for constant 
entropy S is qualitatively different than the graph of G as a function of P shown 
in Fig. 17.3. The essential difference is that in Fig. 17.3 the independent variable, 
P, was intensive, so that the phase transition occurred at a point. In Fig. 17.7 the 
independent variable is extensive, so that it takes on different values in the two phases 
at a first-order transition. 

The first partial derivative of F with respect to V is the negative of the pressure, 
so it is negative. 



(17.18) 
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The second partial derivative of F with respect to V is related to the isothermal 
compressibility, so it is positive. 


d^F \ _ X. 


(17.19) 


We know from the previous discussion of the P — V diagram of the van der 
Waals gas the van der Waals model predicts predicts an unstable region. In Fig. 17.1 
this was found between the points D and F and identified by the positive slope in 
this region, which would imply a negative compressibility. This violates a stability 
condition. 

In terms of the Helmholtz free energy, the unstable region is signaled by a negative 
curvature, which violates the inequality in eq. (17.19). As a consequence, the schematic 
F — V plot exhibits a region of negative curvature between two inflection points, as 
shown in Fig. 17.7. 

As we saw earlier in this chapter, below Tc the homogeneous solution to the van 
der Waals equations is not stable between two values of the volume corresponding to 
the liquid (V;) and the gas (V),). In this region, the Helmholtz free energy is lower 
for coexisting liquid and gas states. The values of F in this coexistence region are 
indicated by the straight dashed line in Fig. 17.7. The line is found as the common 
tangent to the van der Waals curve at the values Vi and Vg. 

It is useful to compare the plot of G vs. P in Fig. 17.3 and the plot of F vs. 1^ in 
Fig. 17.7. The former has an intensive variable on the horizontal axis, and the phase 
transition occurs at a single value of P. The latter has an extensive variable on the 
horizontal axis, and the phase transition occurs over a range of values of V. There are 
many diagrams of thermodynamic potentials as functions of various quantities, but 
they all fall into one of these two catagories. 



Fig. 17.7 Schematic F — V plot for the van der Waals fluid at a fixed temperature T < F. 
The straight dashed line indicates the coexistenee region in which part of the system is liguid in 
equilibrium with the rest of the system in the gas phase. (The true plot has the same features as 
shown here, but it is more difficult to see them clearly due to their relative magnitudes. Generating 
the true plot is left to the exercises.) 
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17.10 Latent Heat 


We all know that it takes heat to boil water. More generally, to cross the coexistence 
curve in Fig. 17.6 requires adding energy to the liquid. If we add a small amount of 
heat (tQ to a system, its entropy changes by an amount dS = dQ/T. Therefore, there 
will be a difference in the entropy between the liquid state corresponding to point G 
and the gas state corresponding to the point C in Fig. 17.1. 

To calculate the difference in entropy, AS, between the liquid and gas phases, 
consider a process by which we integrate the equation 

along the isotherm in Fig. 17.1 from point G to point G. Since the process is isothermal, 
the total heat needed to go from the liquid to the gas phase will be TAS. 

To find the partial derivative in eq. (17.20) we can use a Maxwell relation associated 
with the differential form of the fundamental relation dF = —SdT — PdV + jidN. 


(^\ - (^\ 
V'^^/ T,N V,N 


(17.21) 


With this Maxwell relation, the change in entropy is given by the integral 


AS = 



(17.22) 


Since the partial derivative in eq. (17.22) can be calculated from the van der Waals 
equation of state in eq. (17.3), evaluating the integral is straighforward. 

The energy per particle needed to make a first-order phase transition is called the 
latent heat, denoted by 


N 


(17.23) 


It is also quite common to denote the energy needed to make a first-order phase 
transition for a complete system of N particles as 


L = Ni = TAS 


(17.24) 


17.11 The Clausius-Clapeyron Equation 

It might be supposed that the coexistence curve in the phase diagram in Fig. 17.6 
merely separates the phases and is of no particular significance in itself. However, the 
German physicist Rudolf Clausius, whom we have already encountered as a founder of 
thermodynamics, and the French physicist Benoit Paul Emile Clapeyron (1799-1864) 
found a remarkable equation that links the slope of the coexistence curve with the 
change in volume across the transition and the value of the latent heat. 
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Consider two points along the coexistence curve. Denote the liquid phase at the first 
point by X and the gas phase at the same temperature and pressure by X', with similar 
notation for the liquid, Y, and gas, Y', phases at the other point. Assume that the 
points are very close together, so that the difference in temperature, dT = Ty — Tx, 
and the difference in pressure, dP = Py — Px, are both small. 

Since the liquid and gas phases are in equilibrium, we must have 


htx = T-x' 


(17.25) 


and 


= Mf' 


(17.26) 


which implies that 


Mf — MX = Mf' — MX' (17.27) 

On the other hand, we know from the Gibbs-Duhem equation from eq. (13.9) in 
Section 13.2, that 




Combining these equations, we see that 

dP S' -S AS 
dT ~ V'-V ~ ^ 


(17.28) 


(17.29) 


(17.30) 


Comparing eq. (17.30) with the definition of the latent heat in eq. (17.23), we find the 
Clausius-Clapeyron equation. 


dP _ ex _ L 
dT “ TAV ~ TAV 


(17.31) 


Note that the latent heat is positive by definition, and most materials expand when 
they melt or boil, AV > 0. The slopes of almost all coexistence curves are positive, as 
is the coexistence curve for the van der Waals gas, shown schematically in Fig. 17.6. 

However, ice contracts when it melts, so that AV < 0 and dP/dT < 0 along the ice- 
water coexistence curve. The negative slope required by the thermodynamics analysis 
presented here is confirmed qualitatively and quantitatively by experiment. 
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17.12 Gibbs’ Phase Rule 


Phase diagrams in a general system can be considerably more complicated than 
Fig. (17.6). However, there are limitations on the structure of these diagrams that 
are called ‘Gibbs’ phase rule’. 

To derive Gibbs’ phase rule, consider a general thermodynamic system with K 
components; that is, K different kinds of molecules. Let the number of distinct phases 
in equilibrium with each other be denoted by </>. The variables describing this system 
are T and P, plus the concentrations of each type of particle in each phase. For 
the j-th phase, = iV^7iV|fc = 1,... ,K}, where gives the number of type k 
particles in the j-th phase. Since the total number of particles in the j-th phase is 
]yU) = the sum of the concentrations is unity, 1 = \ there 

are only K — 1 independent concentration variables for each of the (p phases. The total 
number of variables is then 2 -|- (j){K — 1). 

However, the equilibrium conditions on the chemical potentials limit the number 
of independent variables, since the chemical potential of a given component must take 
on the same value for every phase. Letting denote the chemical potential of the 
fc-th component in the j-th phase, we have 


( 1 ) ( 2 ) 

Pk =Pk = 


w 

= pk 


(17.32) 


Since there are K components, this gives a total oiK^p— 1) conditions on the variables 
describing the system. 

Putting these two results together, the total number F of independent variables 
for a general system with K components and p phases is 


F = 2 + P{K - 1) - K{P - 1) 


(17.33) 


or 


F=2+K-p 


(17.34) 


This is Gibbs’ phase rule. 

As an example, consider the case of a simple system containing only a single kind 
of molecule, so that AT = 1. If we consider the boundary between normal ice and liquid 
water {p = 2), we see that the number of independent variables is F = 24-1 — 2 = 1. 
This means that we have one free parameter to describe the boundary between solid 
and liquid. In other words, the two phases will be separated by a line in a phase 
diagram (a plot of P vs. T), and the one parameter will tell us where the system is 
on that line. The same is true for the boundaries between the liquid and vapor phases 
and the solid and vapor phases. 

As a second example, consider the triple point of water, at which three 
phases come together (p = 3). In this case, the number of independent variables is 
F = 2-1-2 — 3 = 0, so that there are no free parameters; coexistence of the three phases 
can occur only at a single point. 

The phase diagram of ice is actually quite complicated, and there are at least ten 
distinct crystal structures at various temperatures and pressures. Gibbs’ phase rule 


Problems 191 


tells US that two different forms of ice can be separated in the phase diagram by a line 
and three phases can coexist at a point, but we cannot have four phases coexisting 
at a point. And, in fact, four phases do not coexist at any point in the experimental 
phase diagram. 


17.13 Problems 


Problem 17.1 

Properties of the van der Waals gas 

1. Just for practice, starting from the expression for the Helmholtz free energy, 
derive the pressure, P = P{T, P, N). 

2. Again starting from the expression for the Helmholtz free energy for the van der 
Waals fluid, derive the energy as a function of temperature, volume, and number 
of particles. 


U = U{T, V, N) 

3. Using the conditions for the critical point. 


dP'\ 

dv) 


1 =0 
T,N 


and 


fd‘^P\ 


= 0 


T,N 


derive expressions for P^ Tc, and 14 as functions of a and b. 

4. Derive the value of the ratio 

PcVc 


NkeTc 

as a pure number, independent of the values of a and b. 


Problem 17.2 

Helmholtz free energy near a phase transition 

Consider a material described by the phase diagram shown below. A quantity of 
this material is placed in a cylinder that is in contact with a thermal reservoir at 
temperature T^. The temperature is held constant, and the pressure is increased as 
shown on the phase diagram, through the point b (on the coexistence curve) to the 
point c. 
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1. Make a qualitative sketch of the pressure P as & function of volume V during 
the process that takes the system from the state a, through the state b, to the 
state c. Label the points on the phase diagram corresponding to states a, b, and 
c. Label the phases. Explain the signs and relative magnitudes of the slopes of 
the lines you draw. Explain any discontinuities in the function or its slope. 

2. Sketch the Helmholtz free energy F as a function of the volume V for this process. 
Again label the phases and the points corresponding to states a, b, and c. Explain 
the relative magnitudes of the slopes and curvatures of the lines you draw. Explain 
any discontinuities in the function or its slope. 


Problem 17.3 

Properties of the van der Waals gas by computation 

1. Write a program to compute and plot E as a function of P at constant T for the 
van der Waals gas. 

2. Modify your program to include a comparison curve for the ideal gas law at the 
same temperature. 

3. Make plots for values of T that are at, above, and below Tc- 

4. For some values of temperature, the plots will have a peculiar feature. What is 
it? 

5. Make a plot of the chemical potential p, as a function of P. You might want to 
include a plot of E vs. P for comparison. 

What does the plot of fi vs. P look like at, above, and below T^? 

The data you need to do a numerical integration of the Gibbs-Duhem relation is 
the same as you’ve already calculated for the first plot in this assignment. You 
just have to multiply the volume times the change in P and add it to the running 
sum to calculate fi vs. P. 


Problem 17.4 

Clausius Clapeyron relatiou 

The slope of a phase boundary in a P — T phase diagram is not arbitrary. The 
Clausius-Clapeyron equation connects it directly to the latent heat and the change in 
volume at the transition. 
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The change in volume between the liquid and the gas, Vgas — Vuquid can be 
approximated by noting that the liquid is much denser than the gas if you are not 
too close to the critical point. We can also approximate the latent heat by a constant 
under the same assumption. 

Find an analytic approximation for the function P(T) along the liquid-gas phase 
boundary. Sketch your answer and compare it to a phase diagram of water that you 
can find on the Web. 


18 

The Nernst Postulate: the Third 
Law of Thermodynamics 


Fourth Law of Thermodynamics: If the probability of success is not almost 
one, then it is damn near zero. 


David Ellis 


In the early part of the twentieth century the German chemist and physicist Walther 
Hermann Nernst (1884-1941) formulated the ‘Nernst Postulate’. It has also become 
known as the Third Law of Thermodynamics, due to its importance. 

The Nernst Postulate: The entropy of a thermodynamic system goes to a constant as the 
temperature goes to zero. 

This postulate has far-reaching consequences. Most directly, it puts important con¬ 
straints on the behavior of real systems at low temperatures. Even more importantly, 
in its universal applicability, it reveals the pervasive influence of quantum mechanics 
on macroscopic phenomena. 

18.1 Classical Ideal Gas Violates the Nernst Postulate 

Since we have only discussed classical statistical mechanics so far in this book, the 
Nernst Postulate might seem rather startling. The results for the classical ideal gas 
that we derived in Part I are completely inconsistent with it. To see this, consider 
eq. (7.2) for the entropy of the classical ideal gas. 



(18.1) 


If we insert the equation of state for the energy. 


U = ^NksT 


( 18 . 2 ) 
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we find an equation for the entropy as a function of temperature. 


S{T, V, N) = kN 



3 ^^ 



+ X 


(18.3) 


Since InT ^ —oo as T —> 0, the entropy of the classical ideal gas does not go to a 
constant at zero temperature. In fact, it will be shown in Chapter 19 that the entropies 
of all classical systems go to —oo at zero temperature. 

The reason for the validity of the Nernst Postulate for real systems lies entirely in 
quantum statistical mechanics, and will be derived in Chapter 22. 


18.2 Planck’s Form of the Nernst Postulate 

Extending the Nernst Postulate, Max Planck made the stronger statement that the 
entropy should not only go to a constant, but that the constant must be zero. This 
is, indeed, the form in which the Nernst Postulate is usually remembered—which is 
unfortunate, since it is not entirely true. Although Nernst’s formulation of his postulate 
is true for all quantum mechanical models and all real systems, Planck’s version is not 
always valid. 

We will return to a justification of the Nernst Postulate and a discussion of Planck’s 
alternative in Chapter 22 on quantum statistical mechanics. 


18.3 Consequences of the Nernst Postulate 

Why is the Nernst Postulate important? One of the main reasons is that it places 
severe limits on the low-temperature behavior of the specific heat and the coefficient 
of thermal expansion. 


18.3.1 Specific Heat at Low Temperatures 

The first consequence of the Nernst Postulate is that the specific heat of anything 
must go to zero at zero temperature. 

To prove this statement, use the connection between a small amount of heat, dQ, 
added to a system and the corresponding change in entropy, dS, in eq. (10.23). 



(18.4) 


We can relate the change of temperature to dQ through the specific heat, which is 
defined in Section 14.4 as 


cx{T) 


N 

T 



(18.5) 


where X stands for either V or P, depending on which is held constant for the 
experiment in question. The heat added is therefore 


dQ = cx{T)dT 


( 18 . 6 ) 
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where we have explicitly indicated the temperature-dependence of the specific heat. 
Putting eqs. (18.4) and (18.6) together and integrating, we find an expression for the 
change in entropy between the temperatures Ti and T 2 , 

S{T2) - S{T,) = I" ^-^dT (18.7) 

Now suppose that the specific heat went to a constant in the limit of zero 
temperature. 

hm^ cjf(T) = Co ( 18 . 8 ) 

Then, for sufficiently low temperatures, we could approximate the specific heat by Cq 
to calculate the change in entropy. 

S{T 2 ) - S(Ti) « [ ' ^dT = Co(lnT 2 - InTi) (18.9) 

JTi J 

In the limit that Ti goes to zero, eq. (18.9) implies that S{T 2 ) — S{Ti) 00 , which 
contradicts the Nernst Postulate. Therefore, the Nernst Postulate requires that the 
specific heat of anything go to zero at zero temperature. 

We have already seen by calculating limr^o S{T) for the classical ideal gas, that 
it violates the Nernst Postulate. This can also be seen from the fact that the specific 
heat at constant volume does not go to zero at zero temperature, but instead has the 
constant value cy = (3/2)^^. 


18.4 Coefficient of Thermal Expansion at Low Temperatures 

Another consequence of the Nernst Postulate is that the coefficient of thermal expan¬ 
sion, defined in Section 14.4 as 

must also vanish at zero temperature. To see why this is true, note that a Maxwell 
relation that can be derived from 


dG = -SdT + VdP + fidN 


(18.11) 


gives us a relationship between a derivative of the entropy and the coefficient of thermal 
expansion. 



T,N 



-Va{T) 


(18.12) 


If the entropy goes to a constant that is independent of pressure, this gives us 
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18.5 Summary and Signposts 

In Part I of this book we developed the foundations of classical statistical mechanics 
based on the existence of atoms and some assumptions about model probabilities. 
We followed Boltzmann in defining the entropy in terms of the logarithm of the 
probability distribution of macroscopic observables in a composite system, and we 
derived a general formula for the entropy of a classical system of interacting particles. 
For the special case of non-interacting particles (the classical ideal gas), we explicitly 
evaluated the entropy as a function of energy, volume, and number of particles. 

In Part II we developed the theory of thermodynamics on the basis of the general 
properties of the entropy that we found in Part I. This led us to develop power¬ 
ful methods for generating exact relationships between measurable thermodynamic 
quantities. 

With this chapter we come to the end of our introduction to thermodynamics, and 
with it the end of Part II of the book. 

We have seen the power of thermodynamics in relating physical properties that, 
at first glance, might seem entirely unrelated to each other. On the other hand, we 
have seen that thermodynamics makes almost no predictions about the actual value 
of any quantity. Indeed, the main exceptions are the predictions for the vanishing of 
the specific heat and the coefficient of thermal expansion at zero temperature. 

Up to this point, the emphasis has been on the concepts in thermal physics and 
their consequences for relationships between measurable quantities. In the remainder of 
the book we will develop more powerful methods for carrying out practical calculations 
in statistical mechanics. 

Part III will discuss classical statistical mechanics, introducing the canonical 
ensemble, which is the workhorse of the field. It will also return to the question of 
irreversibility and the uniqueness of the model probability assumptions we made in 
setting up the foundations of statistical mechanics. 

Part IV will discuss quantum statistical mechanics, including black-body radiation, 
and Bose-Einstein and Fermi-Dirac statistics. It will also show how quantum mechan¬ 
ics leads naturally to the Nernst Postulate. The book will end with an introduction 
to the theory of phase transitions, using the Ising model as an illustration. 

By the end of the book you should have a good foundation in both thermodynamics 
and statistical mechanics—but it will still be only a foundation. The entire field of 
condensed-matter physics will be open to you, with all its complexity and surprises. 
May you enjoy your further explorations in this fascinating area. 
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Ensembles in Classical Statistical 
Mechanics 


In theory, there is no difference between theory and practice. In practice, 
there is. 

Yogi Berra 


This chapter begins Part III, in which we go more deeply into classical statistical 
mechanics. In Chapter 7, eq. (7.46) provided a formal expression for the entropy 
of a general classical system of interacting particles as a function of the energy, 
volume, and number of particles. In principle, this completes the subject of classical 
statistical mechanics, since it is an explicit formula that determines the fundamental 
thermodynamic relation for any classical system of interacting particles. In practice, 
it does not work that way. 

Although eq. (7.46) is correct, carrying out a lO^^-dimensional integral is non¬ 
trivial. It can be done for the ideal gas—which is why that example was chosen for 
Part I—but if the particles interact with each other, the whole business becomes much 
more difficult. 

To make further progress we must develop more powerful formulations of the 
theory. The new expressions we derive will enable us to calculate some properties of 
interacting systems exactly. More importantly, they will enable us to make systematic 
approximations when we cannot find an exact solution. 

The development of the new methods in classical statistical mechanics runs parallel 
to the development of the various representations of the fundamental relation in 
thermodynamics, which was discussed in Chapter 12. For every Legendre transform 
used in finding a new thermodynamic potential, there is a corresponding Laplace 
transform (defined below) that will take us to a new formulation of classical statistical 
mechanics. Just as different Legendre transforms are useful for different problems 
in thermodynamics, different Laplace transforms are useful for different problems in 
statistical mechanics. 

In Part IV we will see that the correspondence between Legendre transforms in 
thermodynamics and ensembles in statistical mechanics carries over into quantum 
statistical mechanics. 
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19.1 Microcanonical Ensemble 

In Chapter 7 we defined the classical microcanonical ensemble by a uniform probability 
distribution in phase space, subject to the constraints that the particles are all in a 
particular volume and that the total energy is constant. The constraint on the positions 
was realized mathematically by limiting the integrals over the positions to the volume 
containing the particles. The constraint on the energy was expressed by a Dirac delta 
function. The Hamiltonian (energy as a function of momentum and position) for a 
system of particles with pair-wise interactions is given by 

Af I ^ |2 N N 
i = l i=l i>3 

The energy constraint is expressed by a delta function. 


5{E - H{p,q)) 


(19.2) 


As in eq. (7.46), the entropy of a classical system of interacting particles is given by 


S{E,V,N) 


klnn{E,V,N) 


fcln 


Jdp5{E-H{p, q)) 


(19.3) 


The integrals Jdq fdp in eq. (19.3) are over the 6iV-dimensional phase space, which, 
unfortunately, makes them difficult or impossible to carry out explicitly for interacting 
systems. In this chapter we will investigate several approaches to the problem that 
enable us to make accurate and efficient approximations for many cases of interest. 
The first approach, which we will consider in the next section, is the use of computer 
simulations. 


19.2 Molecular Dynamics: Computer Simulations 

Computer simulations are remarkably useful for providing us with a good intuitive 
feeling for the properties of ensembles in statistical mechanics. The most natural 
method of simulating the microcanonical ensemble is known as Molecular Dynamics 
(MD). It consists of discretizing time, and then iterating discretized versions of 
Newton’s equations of motion. Since Newton’s equations conserve energy, MD is 
designed to explore a surface of constant energy in phase space, which is exactly 
what we want to do for the microcanonical ensemble. 

For simplicity we will restrict the discussion to a single particle in one dimension. 
This avoids unnecessary notational complexity (indices) while demonstrating most of 
the important features of the method. The extension to more dimensions and more 
particles involves more programming, but no new ideas. 
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Consider a single particle in one dimension, moving under the influence of a 
potential energy V{x). The equations of motion can be written in terms of the 
derivatives of the position and momentum 


dx p 
dt m 
dp dV (x) 

dt dx 


(19.4) 

(19.5) 


The paths in phase space traced out by the solutions to these equations are called 
‘trajectories’. 

The simplest way to discretize this equation is to define a time step St and update 
the positions and momentum at every step. 


x{t + St) = x{t) + — St 

(19.6) 

m 

p{t + St) = p{t) St 

dx 

(19.7) 


While this is not the most efficient method, it is the easiest to program, and it is more 
than sufficient for the simulations in the assignments. 


There are a quite a few ways of implementing discretized versions of Newton’s 
equations—all of which go under the name of Molecular Dynamics. However, 
since our purpose is to understand physics rather than to write the most efficient 
computer program, we will only discuss the simplest method, which is given in 
eqs. (19.6) and (19.7). 


There are two caveats: 

1. Although solutions of the differential equations of motion (19.4) and (19.5) 
conserve energy exactly, the discretized equations (19.6) and (19.7) do not. As 
long as the time step St is sufficiently short, the error is not significant. However, 
results should always be checked by changing the time step to determine whether 
it makes a difference. 

2. Even if energy conservation is not a problem, it is not always true that an MD 
simulation will cover the entire surface of constant energy in phase space. If the 
trajectories do come arbitrarily close to any point on the constant energy surface, 
the system is said to be ergodic. Some examples in the assigned problems are 
ergodic, and some are not. 

The MD simulations in the assignments will show, among other things, the 
distribution of positions in the microcanonical ensemble. The results will become 
particularly interesting when we compare them to the position distributions in the 
canonical ensemble (constant temperature, rather than constant energy), which we 
will explore with the Monte Carlo method in Section 19.11. 
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19.3 Canonical Ensemble 

In this section we will show how to reduce the mathematical difficulties by calculating 
the properties of a system at fixed temperature rather than fixed energy. The result 
is known as the canonical ensemble, and it is an extremely useful reformulation of the 
essential calculations in statistical mechanics. 

The canonical ensemble is the probability distribution in phase space for a system 
in contact with a thermal reservoir at a known temperature. The derivation of the 
canonical probability distribution is essentially the same as that of the Maxwell- 
Boltzmann probability distribution for the momentum of a particle, which was done 
in Subsection 8.3.1. 

We will carry out the calculation in two different ways. First, we will calculate 
the energy distribution between the reservoir and the system of interest, based on the 
results summarized in Chapter 7. Next, we will calculate the probability density for 
the microstates (points in phase space) of the system of interest and show that the 
two calculations are consistent. 


19.3.1 Canonical Distribution of the Energy 

Assume that the system of interest is in thermal contact with a reservoir at tem¬ 
perature T. The entropy of the reservoir is given by an expression like that given in 
eq. (19.3). 

Sr = (19.8) 

We have suppressed the dependence on Vr and Nr in eq. (19.8) to make the notation 
more compact. Assume that the composite system of the thermal reservoir and the 
system of interest is isolated from the rest of the universe, so that the total energy of 
the composite system is fixed. 


Er = E Er (19.9) 

As shown in Chapter 7, eq. (7.31), the probability distribution for the energy in the 
system of interest is given by 

Recall that the most important characteristic of a reservoir is that it is much 
bigger than the system of interest, so that Er >> E and Er >> E. We can use these 
inequalities to make an extremely good approximation to eq. (19.10). Since ^Ir is a 
rapidly varying function of energy, it is convenient to take the logarithm of both sides 
of eq. (19.10). 

In P{E) =lnn{E) + lnnR{ER - E) -In^RiER) (19.11) 


Expand In ^Ir in powers of E. 


In P{E) = In n{E) + In ^/{(Et) -e( ) - In + £> ((E/At)^) (19.12) 

\ oEr J 
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Since the system and the reservoir are at the same temperature in equilibrium, 


T = Tr 


(19.13) 


and 


/? 


1 


Pr 


we have 


OEt ^ keT 


(19.14) 


(19.15) 


Since lnf2fl(£’'r) and In fir are constants, we can replace them in eq. (19.12) with 
a single constant that we will write as In Z. With this change of notation, insert 
eq. (19.15) in eq. (19.12). 


In P{E) =lnn{E) - PE-In Z (19.16) 

Exponentiating eq. (19.16), we find a general expression for the canonical probability 
distribution of the energy. 

P{E) = ^niE)exp{-pE) (19.17) 

In eq. (19.17), Z is simply a normalization ‘constant’. It is constant in the sense 
that it does not depend on the energy E, although it does depend on T, V, and TV, 
which are held constant during this calculation. Using the fact that P{E) must be 
normalized, an expression for Z is easy to obtain. 

Z{T,V,N)= jn{E,V,N)exp{-PE)dE (19.18) 

The integral is over all possible values of the energy E. 

The kind of integral transformation shown in eq. (19.18) is known as a Laplace 
transform. It is similar to a Fourier transform, but the factor in the exponent is real, 
rather than imaginary. 

Z turns out to be extremely important in statistical mechanics. It even has a special 
name: the ‘partition function’. The reason for its importance will become evident in 
the rest of the book. 

The universal use of the letter Z to denote the partition function derives from its 
German name, Zustandssumme: the sum over states. (German Zustand = English 
state.) Eq. (19.18) shows that Z is actually defined by an integral in classical 
mechanics, but in quantum mechanics the partition function can be written as 
a sum over eigenstates, as we will see in Ghapter 22. 
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19.3.2 Canonical Distribution in Phase Space 

This subsection derives the probability density of points in phase space for the 
canonical ensemble. This is more fundamental than the distribution of the energy 
since P{E) can be derived from the probability density in phase space by integrating 
over the delta function S{E — H{p, q)). 

Denote the set of momentum and position variables describing the system of 
interest as {p,q} and the corresponding set for the reservoir as {pR,qR}- The usual 
assumption of a uniform probability distribution then applies to the total phase space 
described by all momentum and position variables from both systems, subject to 
conservation of the total energy, Et- 

To find the probability distribution in the phase space of the system of interest, 
we simply integrate out the variables {prtQr} to obtain the marginal distribution 
P{p, q). This gives us the canonical probability distribution at the temperature of the 
reservoir. 

As shown in Chapter 7, eq. (7.31), the integral over {pr, to} gives us Hr. However, 
since the system of interest has an energy H(j),q), the energy in the reservoir is 
Et — H(jp, q). The resultant equation for the probability distribution in the phase 
space of the system of interest can be written as 


P{p,q) 


HR{ET-H{p,q)) 

Ht{Et) 


(19.19) 


Since the reservoir is much larger than the system of interest, Er >> H(p,q). 
Taking the logarithm of both sides of eq. (19.19) and expanding InH/j in powers of 
H{p,q)/E r^ we find 

In P(p, q) = In Hr{Et) - H{p, q) .^Hr{Et) - In Ht{Et) + ■■■ (19.20) 

Recalling that 

P = Pr=-^Hr{Et) (19.21) 

and noting that only the second term on the right in eq. (19.20) depends on p or q, 
we can write 


In P(p, q) = -(3H{p, q) - In Z{T, V, N) (19.22) 

or 

P{p, q) = exp[-/3i7(p, q)] (19.23) 

Z(1 , 

The function Z(T, V, A^) is given by the normalization condition. 

Z{T,V,N) = Jdq Jdp exp[-PH{q,p)] (19.24) 
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The integral in eq. (19.24) is over all of phase space. 

The only approximation in eq. (19.23) is the assumption that the reservoir is 
very large in comparison with the system of interest, which is usually an excellent 
approximation. 

We still have to relate Z(T, V, N) to the partition function Z{T, V, N), which we 
will do in the next section. 

19.4 The Partition Function as an Integral over Phase Space 

If we combine the general equation for Q, eq. (7.30), 

n{E,V,N) = j^^ jdq jdp5{E-H{q,p)) (19.25) 

with the definition of the partition function in eq. (19.18), 

Z{T,V,N) = jn{E)exp{-PE)dE (19.26) 

and carry out the integral over the delta function, we find a very useful expression for 
the partition function in terms of an integral over phase space. 

Z = JdEj-^^—^ jdq JdpS{E - H{q,p))exp{-f3E) 

= y^'3' J^P JdES{E - H{q,p))exp{-PE) 

= j<^d Jdpexp[-PH{q,p)] (19.27) 


The final expression in eq. (19.27) is probably the most important equation for 
practical calculations in classical statistical mechanics. You should be able to derive 
it, but it is so important that you should also memorize it. 

Comparing eq. (19.27) to eq. (19.24), we see that Z{T,V,N) must be given by 

Z{T,V,N) = h^^mZ (19.28) 

The canonical probability density in phase space is then 

P{P, d) = I ^ exp[-PH{p, g)] (19.29) 

19.5 The Liouville Theorem 

Points in phase space move as functions of time to trace out trajectories. This rather 
obvious fact raises disturbing questions about the stability of the canonical distribution 
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as a function of time. If a system is described by eq. (19.29) at a given time, will it 
still be described by the same distribution at a later time? 

It is certainly not the case that an arbitrary, non-equilibrium probability distri¬ 
bution in phase space remains unchanged as time passes. We can see macroscopic 
changes in non-equilibrium systems, so the probability distribution must change. 

Fortunately, for the canonical ensemble, the probability distribution is stable and 
does not change with time. This statement is based on the Liouville theorem, named 
after the French mathematician Joseph Liouville (1809-1882). 

To prove the Liouville theorem, consider the probability density in phase space 
P{p,q). Since points in phase space represent microscopic states, they can neither be 
created nor destroyed along a trajectory. We can then regard the points as abstract 
‘particles’ moving in a 67V-dimensional space, with P{p, q) corresponding to the density 
of such points. As for all gases of conserved particles, the points in phase space must 
obey a continuity equation. 


dPjp, q) 
dt 


+ V • {P{p,q)v) = 0 


The gradient in eq. (19.30) is defined by the vector 


V - <) A A 
dqj ’ dpj 


j = l,...,3iV 


and the fiWdimensional ‘velocity’ is defined by 


{ ^ ^ l,...,3iv| ={qj^p^\j = 1,...,3A^}, 


(19.30) 


(19.31) 


(19.32) 


where we have used a dot to indicate a partial derivative with respect to time. 
The continuity equation then becomes 


37V r 


dP 


^^iP{p.q)q,) + ^^iPip,q)p,) 


= 0 


(19.33) 


or 


— + -P A 

fit ^ 


J=1 


dq-j , dpj 

3N 

dP . dP 

dpj 

+ E 

i=i 



= 0 


(19.34) 


Recall in classical mechanics that the time development of a point in phase space is 
given by Hamilton’s equations, first derived by the Irish physicist and mathematician 
Sir William Rowan Hamilton (1805-1865). 


dH 

dH 


(19.35) 


(19.36) 
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H = H(j), q) is, of course, the Hamiltonian. These two equations give us an interesting 
identity. 


dq, _ d^H 
dqj dqjdpj 


d^H 


dPj 

dpj 


dpj dqj 

Inserting eq. (19.37) into eq. (19.34), we find that it simplifies to 


(19.37) 


3Af 


dP V- 


dP 


dP 


i=i 




= 0 


(19.38) 


To complete the proof of Liouville’s theorem, we must only insert eq. (19.38) into 
the equation for the total derivative of P{p, q) with respect to time. 


dP 

dt 


dP ^ 


1=1 


dP 

dq. 




dP 

dp. 


■Pj 


= 0 


(19.39) 


The interpetation of eq. (19.39) is that the probability density in the neighborhood 
of a moving point remains constant throughout the trajectory. 

The application of Liouville’s theorem to the canonical ensemble rests on the 
property that the probability density depends only on the total energy. Since the 
trajectory of a point in phase space conserves energy, the canonical probability 
distribution does not change with time. 


19.6 Consequences of the Canonical Distribution 

The probability density, P{E), for the energy in the canonical ensemble is extremely 
sharply peaked. The reason is that P{E) is the product of two functions that both 
vary extremely rapidly with energy: fl(E) increases rapidly with increasing E, while 
exp{—/3E) goes rapidly to zero. 

To see that this is true, first recall that id{E) for the ideal gas has an energy 
dependence of the form 


n{E) oc E^ 


(19.40) 


where / = SN/2. For more general systems, / is usually a relatively slowly varying 
function of the energy, but remains of the same order of magnitude as the number 
of particles N —at least to within a factor of 1000 or so—which is sufficient for our 
purposes. 

If we approximate / by a constant, we can find the location of the maximum of 
P{E) by setting the derivative of its logarithm equal to zero. 


dliiP{E) 


^[lnn{E)-PE-lnZ] = 


p = o 


(19.41) 
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The location of the maximum of P{E) gives the equilibrium value of the energy. 


E 


eq — 


/ 

/3 


IksT 


(19.42) 


The width of the probability distribution for the energy can be found from the 
second derivative if we approximate P{E) by a Gaussian function, as discussed in 
connection with applications of Bayes’ theorem in Section 5.5. 


d'^\nP{E) 


A [/ 

dE E 


P 


E^ a% 


(19.43) 


Evaluating the expression for the second derivative in eq. (19.43) a,t E = Egqui, we 
find the width of P{E). 


(Te = 


Eeq 

77' 


(19.44) 


When / is of the order N, and N w 10^° or larger, the relative width of the prob¬ 
ability distribution is about 10”^° or smaller. This is much smaller than the accuracy 
of thermodynamic measurements. Note that the 1/VN dependence is the same as 
we found for the width of the probability distribution for the number of particles in 
Chapter 4. 


19.7 The Helmholtz Free Energy 

The fact that the width of the probability distribution for the energy is roughly pro¬ 
portional to l/V N in the canonical ensemble has an extremely important consequence, 
which is the basis for the importance of the partition function. 

Consider eq. (19.16) for the logarithm of the canonical probability density for the 
energy. 


In P{E) =-(3E +lnn{E)-In Z (19.45) 

From eq. (7.46), we know that S = ks Infl, so we can rewrite eq. (19.45) as 

\nZ = -I3E + S/kB-lnP{E) (19.46) 

= -P{E-TS)-\nP{E) 

= -(3F -\nP{E) 


Since the probability distribution P{E) is normalized and its width is roughly 
proportional to 1 /VN , its height must be roughly proportional to ^/N. If we evaluate 
eq. (19.46) at the maximum of P{E), which is located a,t E = E^q, the term In P{E) 
must be of the order (l/2)lniV. However, the energy E, the entropy S, and the free 
energy F = E — TS are all of order N. If N « 10^°, then 


Ir^PjEeq) 

PE 


IniV 


5 X 10"^® 


N 


(19.47) 
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This means that the last term in eq. (19.46) is completely negligible. We are left with 
a simple relationship between the canonical partition function and the Helmholtz free 
energy. 

In Z{T, V, N) = -PF{T, V, N) (19.48) 

or 

F{T,V,N) =-kBTlnZ{T,V,N) (19.49) 

To remember this extremely important equation, it might help to recast the familiar 
equation 


S=kBlnn (19.50) 

in the form 

H = exp[S/kB] (19.51) 

In carrying out the Laplace transform in eq. (19.18), we multiplied by H the factor 
exp[—PE] before integrating over E. The integrand in eq. (19.18) can therefore be 
written as 


nexp[—PE] = exp[—/3E + •S’/fcs] 

= exp[—f3{E — TiS)] 

= exp[-/?F] (19.52) 


19.8 Thermodynamic Identities 


Since the postulates of thermodynamics are based on the results of statistical mechan¬ 
ics, it should not be too surprising that we can derive thermodynamic identities directly 
from statistical mechanics. 

For example, if we take the partial derivative of the logarithm of the partition 
function in eq. (19.27) with respect to (3, we find that it is related to the average 
energy. 


^InZ 

dp 


1 dZ 
'Z~dp 

exp[-/3i7(g,p)] 




1 1 
zlWm 


Idq dp H{q,p) exp[-PH{q,p)] 


^ Jdq JdpH{q,p)exp[-PH{q,p)] 
JdqJdp exp[-PH{q,p)] 

= -{E) 


(19.53) 
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Since we know that InZ = —(3F, eq. (19.53) is equivalent to the thermodynamic 
identity 


U={E) 


djPF) 

8/3 


(19.54) 


which was proved as an exercise in Chapter 14. 


19.9 Beyond Thermodynamic Identities 

Because statistical mechanics is based on the microscopic structure of matter, rather 
than the phenomenology that led to thermodynamics, it is capable of deriving rela¬ 
tionships that are inaccessible to thermodynamics. An important example is a general 
relationship between the fluctuations of the energy and the specific heat. 

Begin with the expression for the average energy from eq. (19.53). 


U={E) 

= ^ jdp H{q,p) exp[-PH{q,p)] 

^ JdqJdpH{q,p)exp[-l3H{q,p)] 

JdqJdp exp[-PH{q,p)] 


Note that the dependence on the temperature appears in only two places in eq. (19.55): 
once in the numerator and once in the denominator. In both cases, it appears in an 
exponent as the inverse temperature (3 = l/ksT. It is usually easier to work with /3 
than with T in statistical mechanical calculations. To convert derivatives, either use 
the identity 


(i/9 (i/9 / 1 \ —1 

FT ~ df VfcBT ) ~ ksT"^ 

or the identity 

dT _ d ( I \ _ -1 
dfd dp yksp) ksP"^ 


(19.56) 


(19.57) 


The specific heat per particle at constant volume can then be written as 




N dT 


1 -1 8 

NksT"^ W 


(E) 


(19.58) 


We can now take the partial derivative of {E) in eq. (19.55) 

— (E)= — \ !<^PH{q,p) exp[-PH{q, p)] ' 

8P 8P _ jdq Jdp exp[—PF[{q,p)] 

^ Jdq jdp H{q,p){-H{q,p))exp[-PH{q,p)\ 

JdqJdp exp[-PH{q,p)\ 
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Jdq Jdp H{q,p) exp[-PH{q,p)] Jdq Jdp {-H{q,p)) exp[-PH{q,p)] 
[fdqfdp exp[-PH{q,p)]]'^ 

= -{E^) + {Ef (19.59) 

and use eq. (19.58) to find the specific heat. 

^ ^ ~Nk^ 

In relating the fluctuations in the energy to the specific heat, eq. (19.60) goes 
beyond thermodynamics. It gives us a direct connection between the microscopic 
fluctuations and macroscopic behavior. This relationship turns out to be extremely 
useful in computational statistical mechanics, where it has been found to be the most 
accurate method for calculating the specific heat from a computer simulation. 

It should be clear from the derivation of eq. (19.60) that it can be generalized 
to apply to the derivative of any quantity of interest with respect to any parameter 
in the Hamiltonian. The generality and flexibility of this technique has found many 
applications in computational statistical mechanics. 


19.10 Integration over the Momenta 

A great advantage of working with the canonical partition function is that the integrals 
over the momenta can be carried out exactly for any system in which the forces do 
not depend on the momenta. This, unfortunately, eliminates systems with moving 
particles in magnetic fields, but still leaves us with an important simplification for most 
systems of interest. As an example, we will consider the case of pairwise interactions 
for simplicity, although the result is still valid for many-particle interactions. 

If E[(p, q) only depends on the momenta through the kinetic energy, the partition 
function in eq. (19.27) becomes 


Z = 




h^N]S[\ 

1 

1 


dq exp 


N 




\pjf 


N N 


2„ ■ 

i=i 1=1*>1 



|2 


N N 

/ dp exp 

I3T 

^ 2m 
1=1 

/ dq exp 

1=1 i>j 




y {2nmkBT)^^^^ jdq exp 


N N 

1=1 i>j 


(19.61) 


where we have used jd = l/ksT in the last line of eq. (19.61). The momenta have been 
integrated out exactly. 
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19.10.1 The Classical Ideal Gas 

For the ideal gas, the interactions between particles vanish [(j)(ri,rj) = 0], and we can 
rederive the main results of Part I in a few lines. The integrals remaining in eq. (19.61) 
give a factor of V for each particle, so that 

^ ^ fe3Wjv! Jdq exp(O) 

= (19.62) 

The logarithm of eq. (19.62) gives the Helmholtz free energy F = —ksTlnZ of 
the classical ideal gas as a function of temperature, volume, and number of particles, 
which is a representation of the fundamental relation and contains all thermodynamic 
information. It is left as an exercise to confirm that the explicit expression for F agrees 
with that derived from the entropy of the classical ideal gas in Part I. 

19.10.2 Computer Simulation in Configuration Space 

Eq. (19.61) is used extensively in Monte Carlo computer simulations because it 
eliminates the need to simulate the momenta and therefore reduces the required 
computational effort. Monte Carlo computations of the thermal properties of models 
in statistical mechanics will be discussed in the next section. 

19.11 Monte Carlo Computer Simulations 

Just as the Molecular Dynamics method is the most natural way to simulate a 
microcanonical ensemble (constant energy), the Monte Carlo (MC) method is the 
most natural way to simulate a canonical ensemble (constant temperature). The MC 
method ignores the natural trajectories of the system in favor of a random sampling 
that reproduces the canonical probability distribution. In practice, applications of MC 
to classical statistical mechanics usually simulate the equilibrium canonical probability 
distribution for the configurations after the momenta have been integrated out. 

Peq{q) = ^exp[-/3P(g)] (19.63) 

In eq. (19.63), which follows directly from eq. (19.29) upon integrating out the 
momenta, the normalization constant Q plays a role very similar to the partition 
function. It is defined by 

Q = j eyLp[-l3V{q)]dq (19.64) 

where the integral goes over the allowed volume in configuration space. 

To implement an MC simulation, we first construct a stochastic process for prob¬ 
ability distributions defined in configuration space. A stochastic process is a random 
sequence of states (configurations in our case) in which the probability of the next 
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state depends on the previous states and the parameters of the model (temperature, 
interaction energies, and so on). Our goal is to find a stochastic process for which 
an arbitrary initial probability distribution will evolve into the desired equilibrium 
distribution. 

lim P{q,t) = Peq{q) = ^ exp[- /3V(q)] (19.65) 

i—►€» Q 

Once we have such a process we can use it to generate states with the equilibrium 
probability distiribution and average over those states to calculate thermal properties. 

If the next state in a stochastic process does not depend on states visited before 
the previous one, it is called a Markov process. Since we will see below that they are 
particularly convenient for simulations, we will consider only Markov processes for MC 
simulations. 

For a given Markov process, we can define a conditional probability for a particle 
to be in configuration q' at time t + 6t if it was in configuration q at time t. We will 
denote this conditional probability as W{q'\q). (It is also called a transition probability 
and is often denoted by W{q^ q') to indicate the direction of the transition.) Since 
the transition from one state to the next in a Monte Carlo simulation is discrete, 5t is 
called the time step and is not and infinitesimal quantity. 

The change in the probability density P((j, t) for q between times t and t + St is 
then given by the ‘Master Equation’, 

P{q,t + St) - P{q,t) = J [W{q\q')P{q',t) - W{q'\q)P{q,t)] dq' (19.66) 

where the integral in eq. (19.66) goes over all allowed configurations q'. The term 
W{q\q')P{q', t) in the integrand of eq. (19.66) represents the increase in the probability 
of state q due to transitions from other states, while W{q'\q)P{q,t) represents the 
decrease in the probability of state q due to transitions out of that state. Obviously, 
the conditional (or transition) probabilities must be normalized, so that 

Jw{q'\q)dq' = 1 (19.67) 

for all q. 

A necessary condition for the simulation to go to equilibrium as indicated in 
eq. (19.65) is that once the probability distribution reaches Peq{q), it stays there. In 
equilibrium, P{q,t) = Peq{q) is independent of time, and the master equation becomes 

Peq{q) - Peqiq) = « = / {q\q')Peqiq') - W{q'\q)Peq{q)] dq' (19.68) 

Although the Markov process will remain in equilibrium as long as the integral in 
eq. (19.68) vanishes, the more stringent condition that the integrand vanish turns out 
to be more useful. 


W{q\q')Peqiq') - Wiq'\q)Peqiq) = 0 


( 19 . 69 ) 
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Eq.(19.69) is called the condition of ‘detailed balance’, because it specifies that the 
number of transitions between any two configurations is the same in both directions. 

It can be shown that if there is a finite sequence of transitions such that the system 
can go from any configuration to any other configuration with non-zero probability 
and that detailed balance holds, the probability distribution will go to equilibrium as 
defined in eq. (19.65). After the probability distribution has gone to equilibrium, we 
can continue the Markov process and use it to sample from Peq{q)- 

The condition that there is a finite sequence of transitions such that the system can 
go from any configuration to any other configuration with non-zero probability is 
called ‘ergodicity’. This terminology is unfortunate, because the word ‘ergodicity’ 
is also used to refer to the property that dynamical trajectories come arbitrarily 
close to any given point on an energy surface, as discussed in connection with MD 
simulations. It is much too late in the history of the subject to produce words that 
clearly distinguish these two concepts, but we may take some consolation in the 
fact that one meaning is used only with MC, and the other only with MD. 


The great thing about the condition of detailed balance in eq. (19.69) is that we 
are able to decide what the conditional probabilities W{q'\q) will be. For simulating 
a canonical ensemble, we want to use this power to create a Markov process that 
produces the canonical distribution. 

lim P{q,t) = Peg{q) = ^exp[-pV{q)] (19.70) 

t—>c» ^ Q 

As long as the conditional (transition) probabilities do not vanish, the condition 
of detailed balance can be written as 

or 

(19.72) 

where AE = V{q') — V{q). Note that the ‘partition function’, Q, has canceled out of 
eq. (19.72)—which is very convenient, since we usually cannot evaluate it explicitly. 
There are many ways of choosing the W{q'\q) to satisfy eq. (19.72). The oldest and 
simplest is known as the Metropolis algorithm. 

Metropolis algorithm: 

A trial configuration qtrtai is chosen at random. 

• If AE < 0, the new state is q' = qtriai- 

• If AE > 0, the new state is q' = qtriai with probability exp{—/3AE), 
and the same as the old state q' = q with probability 1 — exp{—(3AE). 
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It is easy to confirm that the Metropolis algorithhm satisfies detailed balance. 


There are many things that have been swept under the rug in this introduction 
to the Monte Carlo method. Perhaps the most serious is that there is nothing in 
eq. (19.70) that says that convergence will occur within your computer budget. 
Slow MC processes can also make acquiring data inefficient, which has prompted 
a great deal of effort to go into devising more efficient algorithms. There are also 
many methods for increasing the amount of information acquired from MC (and 
MD) computer simulations. Fortunately, there are a number of excellent books on 
computer simulation methods available for further study. 


We will again restrict our examples in the assignments to a single particle in one 
dimension, as we did in Section 19.2 for Molecular Dynamics simulations. Remember 
to compare the results of the two simulation methods to see the different kinds of 
information each reveals about the thermal properties of the system. 


19.12 Factorization of the Partition Function: the Best Trick 
in Statistical Mechanics 

The key feature of the Hamiltonian that allowed us to carry out the integrals over the 
momenta in eq. (19.61) and the integrals over the coordinates in eq. (19.62) is that 
the integrand exp[—f3H] can be written as a product of independent terms. Writing 
a 3iV-dimensional integral as a one-dimensional integral raised to the 37V-th power 
takes a nearly impossible task and makes it easy. 

We can extend this trick to any system in which the particles do not interact with 
each other, even when they do interact with a potential imposed from outside the 
system. Consider the following Hamiltonian. 

N |-,|2 N 

i=i i=i 

The partition function for the Hamiltonian in eq. (19.73) separates neatly into a 
product of three-dimensional integrals. 


Z = 


{2TtmkBTf^^‘^ Jdq exp 


h^^Nl 

1 

1 


N 




i=i 


N 


(2TrmkBT)^^^^ dq exp —P^j{rj) 
j=i 

N I- 

hSNNl jdrj exp 


(19.74) 


i=i' 
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Eq. (19.74) has reduced the problem from one involving an integral over a SN- 
dimensional space, as in eq. (19.61), to a product of N three-dimensional integrals. 

When the three-dimensional integrals are all the same, eq. (19.74) simplifies 
further. 


Z = 


1 


h^^Nl 


(27TmkBT)^^^^ ( dr I exp —/3^i(ri) 


N 


(19.75) 


Since a three-dimensional integral can always be evaluated, even if only numerically, 
all problems of this form can be regarded as solved. 

This trick is used repeatedly in statistical mechanics. Even when it is not possible 
to make an exact factorization of the partition function, it is often possible to make a 
good approximation that does allow factorization. 

Factorization of the partition function is the most important trick you need for 
success in statistical mechanics. Do not forget it! 


19.13 Simple Harmonic Oscillator 

The simple harmonic oscillator plays an enormous role in statistical mechanics. The 
reason might be that many physical systems behave as simple harmonic oscillators, or 
it might be just that it is one of the few problems we can solve exactly. It does provide 
a nice example of the use of factorization in the evaluation of the partition function. 


19.13.1 A Single Simple Harmonic Oscillator 

First consider a single, one-dimensional, simple harmonic oscillator (SHO). The Hamil¬ 
tonian is given by the following equation, where the subscript 1 indicates that there 
is just a single SHO. 

H. = ^ (19.70 

The partition function is found by simplifying eq. (19.27). The integrals over x and 
p are both Gaussian, and can be carried out immediately. 




i (27rmfcBT)^^^ {2TtkBT/ 
1 /m\i/2 

^ \k) 

1 

fdhio 


(19.77) 
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where 



(19.78) 


is the angular frequency of the SHO. 

As we have mentioned earlier, the factor of h has no true significance within classical 
mechanics. However, we will see in Section 23.10 that the result in eq. (19.77) does 
coincide with the classical limit of the quantum simple harmonic oscillator given in 
eq. (23.67). 

19.13.2 N Simple Harmonic Oscillators 

Now consider a macroscopic system consisting of N one-dimensional simple harmonic 
oscillators. 



(19.79) 


Note that the spring constants Kj and the masses nij might all be different. 

Since the oscillating particles are localized by the harmonic potential, we do not 
need to consider exchanges of the particles with another system. Therefore, we do 
not include an extra factor of 1/A^! to appear in the partition function. Actually, 
since N is never varied in this model, a factor of 1/A^! would not have any physical 
consequences. However, excluding it produces extensive thermodynamic potentials 
that are esthetically pleasing. 

After integrating out the momenta, the partition function can be written as 




1 ’ 
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where 



(19.81) 


is the angular frequency of the j-th SHO. 

If all N SHO’s have the same frequency, the partition function simplifies further. 



(19.82) 


19.14 Problems 


Problem 19.1 

Two one-dimensional relativistic particles 

Consider two one-dimensional relativistic ideal-gas particles with masses confined 
to a one-dimensional box of length L. Because they are relativistic, their energies are 
given by Ea = \pa\c and Ea = \pb\c. 

Assume that the particles are in thermal equilibrium with each other, and that the 
total kinetic energy is E = Ea + Eb- Use the usual assumption that the probability 
density is uniform in phase space, subject to the constraints. 

Calculate the probability distribution P{Ea) for the energy of one of the particles. 

Problem 19.2 

Classical simple harmonic oscillator 

Consider a one-dimensional, classical, simple harmonic oscillator with mass m and 
potential energy 



The SHO is in contact with a thermal reservoir (heat bath) at temperature T. 
Calculate the classical partition function, Helmholtz free energy, and energy. 

Problem 19.3 

Canonical ensemble for anharmonic oscillators 

We have shown that the classical canonical partition function is given by 
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Consider N non-interacting (ideal gas) particles in an anharmonic potential, 

-(ixr + i^r + izD 

r 


where r is a positive constant. 

The full Hamiltonian can be written as: 

N 2 JV 3^ 

n=l j=l 

Calculate the energy and the specific heat using the canonical partition function. 

Hint: The integrals over the momenta shouldn’t cause any problems, since they 
are the same as for the ideal gas. However, the integrals over the positions require a 
trick that we used in deriving the value of the gaussian integral. 

Another hint: You might find that there is an integral that you can’t do, but don’t 
need. 


Problem 19.4 

Partition function of ideal gas 

1. Calculate the partition function and the Helmholtz free energy of the classical 
ideal gas using the canonical ensemble. 

2. Starting with the entropy of the classical ideal gas, calculate the Helmholtz free 
energy and compare it with the result from the canonical ensemble. 


Problem 19.5 

Molecular Dynamics (MD) simulation of a simple harmonic oscillator 

This assignment uses the Molecular Dynamics method, which simply consists of 
discretizing Newton’s equations of motion, and then iterating them many times to 
calculate the trajectory. 

Write a computer program to perform a Molecular Dynamics (MD) computer 
simulation of a one-dimensional simple harmonic oscillator (SHO). 

[You can take A" = 1 in answering the questions.] 

Include a plot of the potential energy, including a horizontal line indicating the 
initial energy. [You will also do Monte Carlo (MC) simulations of the same models 
later, so remember to make your plots easy to compare.] 

The program should read in the starting position, the size of the time step, and 
the number of iterations. It should then print out a histogram of the positions visited 
during the simulation. 


222 Ensembles in CIsssical Statistical Mechanics 


Your program should also print out the initial and final energy as a check on 
the accuracy of the algorithm. Don’t forget to include the kinetic energy in your 
calculation! 

1. For a starting value of x = 1.0. Try various values of the size of the time step to 
see the effect on the results. 

What happens when the time step is very small or very large? 

2. Choose X = 1.0 and run the program for an appropriate value of dt. Print out 
histograms of the positions visited. [Later, you will compare them with you results 
for MC 


Problem 19.6 

Molecular Dynamics (MD) simulation of an anharmonic oscillator 

This assignment again uses the Molecular Dynamics method. Only minor modifications 
of your code should be necessary, although I would recommend using functions to 
simplify the programming. 

Write a computer program to perform a Molecular Dynamics (MD) computer 
simulation of a one-dimensional particle in the following potential. 

H = Ax^ + Bx^ + Cx^ 

[The values of A, B, and C should be read in or set at the beginning of the program. 
Boltzmann’s constant can again be taken as ks = 1, but the variable should also be 
included explicitly in the program.] 

Include a plot of the potential energy, including a horizontal line indicating the 
initial energy. 

The program should read in the starting position, the size of the time step, and 
the number of iterations. It should then print out a histogram of the positions visited 
during the simulation. 

Your program should also print out the initial and final energy as a check on 
the accuracy of the algorithm. Don’t forget to include the kinetic energy in your 
calculation! 

1. Carry out simulations with the values A = —1.0, B = 0.0, and C = 1.0. 

First try various starting value of the position x. Choose the starting positions 
to adjust the (constant) total energy to take on “interesting” values. Part of the 
problem is to decide what might be interesting on the basis of the plot of the 
potential energy. 

2. Now change the value of B to —1.0, while retaining A = —1.0 and C = 1.0. Again 
look for interesting regions. 


Problem 19.7 

Molecular Dynamics (MD) simulation of a chain of simple harmonic oscil¬ 
lators 

This assignment again uses the Molecular Dynamics method, but this time we will 
simulate a chain of particles connected by harmonic interactions. 
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Write a computer program to perform a Molecular Dynamics (MD) computer 
simulation of the following Hamiltonian. 



Take the particle at xq = 0 to be fixed. 

1. For various (small) values of N, try different values of the time step while 
monitoring the change in total energy during the simulation. See if a good choice 
of time step agrees with what you found for a single particle. 

2. For iV = 1, make sure that you get the same answer as you did for HW2. [Just 
confirm that you did it; you don’t have to hand it in.] 

3. For N = 2 and 3, and random starting values of position and momentum, what 
distributions do you get for xi and xn-i — xn- 2 ^ 

4. For N = 10, start particle 1 with an initial position of a;[l] = 1.0, let the initial 
positions of all other particles be 0.0, and let the initial value of all momenta 
be 0.0. What distribution do you get for xj — Xj-i for whatever value of j you 
choose? 

5. Which results resemble what you might expect if your chain were in equilibrium 
at some temperature? Can you estimate what that temperature might be? 

Problem 19.8 

Molecular Dynamics (MD) simulation of a chain of simple harmonic oscil¬ 
lators - AGAIN 

This assignment again uses the MD program you wrote to simulate a chain of particles 
connected by harmonic interactions. 



[The particle at xq = 0 is fixed.] 

1. You know that the average energy of a single simple harmonic oscillator at 
temperature T is given by 


U={H) = keT, 


so that the specific heat is 


c = ks- 


Demonstrate analytically that the energy per particle and the specific heat per 
particle are the same for a harmonic chain and a single SHO without using a 
Fourier transform of the Hamiltonian. 

2. Modify your program to calculate the expected kinetic energy per particle from 
the initial (randomly chosen) energy. Then use the average kinetic energy to 
predict an effective temperature. 
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3. Modify your program to find the contributions to the specific heat from the 
fluctuations of the potential energy under the assumption that the kinetic degrees 
of freedom act as a thermal reservoir at the predicted effective temperature. 

[The contributions to the specific heat from the momenta are fc_B/2 per particle.] 

4. Calculate the effective temperature and the specific heat for various lengths of 
the harmonic chain. To what extent are the relationships between the energy and 
the effective temperature and the fluctuations and the specific heat confirmed by 
your data? 


Problem 19.9 

Not-so-simple harmonic oscillators 

Consider a one-dimensional, generalized oscillator with Hamiltonian 

H = K\x\°‘ 


with the parameter a > 0. 

Calculate the energy and specific heat as functions of the temperature for arbitrary 
values of a. 


Problem 19.10 

Monte Carlo simulations of not-so-simple harmonic oscillators 

Consider a one-dimensional, generalized oscillator with Hamiltonian 

H = A\xr 

1. Modify your program for MC simulations to simulate the the potential energy 
given in this problem. For values of a = 0.5,1.0,, abd 4.0, run MC simulations 
and compare the results to your theoretical results for the heat capacity from an 
earlier assignment. 


Problem 19.11 

Relativistic particles in a gravitational field 

Consider N relativistic ideal-gas particles at temperature T. The particles are confined 
to a three-dimensional box of area A and 0 < z < oo. Because they are relativistic, 
the Hamiltonian is given by 

N N 

H (p, l) = X! 1*^ + X! 

i=i i=i 

1. Find the probability distribution for the height of a single particle, i.e.: P{zi). 

2. Find the average height of a single particle. 
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3. In what way did your answer for the average height depend on the relativistic 
nature of the particles? 

4. Find the probability distribution for the energy of a single particle, Ei = |pi|c + 
mgzi, including the normalization constant. 


Problem 19.12 

A classical model of a rubber band 

The rubber band is modeled as a one-dimensional polymer, that is, a chain of N 
monomers. Each monomer has a fixed length i5, and the monomers are connected 
end-to-end to form the polymer. Each monomer is completely free to rotate in two 
dimensions (there are no steric hinderances from the other monomers). A microscopic 
state of the rubber band is therefore described by the set of angles 

9 = {0^\n=l,2,...,N} 

describing the orientations of the monomers. [You can decide whether to define the 
angles with respect to the orientation of the previous monomer, or according to a fixed 
frame of reference. However, you must say which you’ve chosen!] 

The rubber band (polymer) is used to suspend a mass M above the floor. One 
end of the rubber band is attached to the ceiling, and the other end is attached to the 
mass, so that the mass is hanging above the floor by the rubber band. For simplicity, 
assume that the rubber band is weightless. You can ignore the mass of the monomers 
and their kinetic energy. 

The whole system is in thermal equilibrium at temperature T. 

1. Calculate the energy of the system (polymer plus mass) for an arbitrary micro¬ 
scopic state 9 of the monomers (ignoring the kinetic energy). 

2. Find an expression for the canonical partition function in terms of a single, one¬ 
dimensional integral. 

3. Find an expression for the average energy U. 

4. For very high temperatures, find the leading term in an expansion of the average 
energy U. 
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Classical Ensembles: Grand 
and Otherwise 


To create man was a quaint and original idea, but to add the sheep was 
tautology. 

Mark Twain (Samuel Clemens) 


Although the canonical ensemble is the work horse of classical statistical mechanics, 
several other ensembles often prove convenient. The most important of these is the 
‘grand canonical ensemble’, which will be discussed in this chapter. This discussion 
will also serve as an introduction to the very important use of the quantum grand 
canonical ensemble, which will play a prominent role in the theories of boson and 
fermion fluids in Part IV. 


20.1 Grand Canonical Ensemble 

The physical situation described by the grand canonical ensemble is that of a system 
that can exchange both energy and particles with a reservoir. As usual, we assume 
that the reservoir is much larger than the system of interest, so that its properties are 
not significantly affected by relatively small changes in its energy or particle number. 
Note that the reservoir must have the same type (or types) of particle as the system 
of interest, which was not a requirement of the reservoir for the canonical ensemble. 

Because the system of interest and the reservoir are in equilibrium with respect 
to both energy and particle number, they must have the same temperature T = Tr 
and chemical potential p, = pn. In thermodynamics, this situation would correspond 
to the thermodynamic potential given by the Legendre transform with respect to 
temperature and chemical potential. 

U[T,p\=U -TS - pN (20.1) 

The function U[T,p\ is sometimes referred to as the ‘grand potential’. Since Legendre 
transforms conserve information, the grand potential is another representation of 
the fundamental relation and contains all thermodynamic information about the 
system. 
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For extensive systems, Euler’s equation, eq. (13.4), holds, and U[T,fi] has a direct 
connection to the pressure. 


U[T, fi] = U -TS - fiN = -PV 


( 20 . 2 ) 


While the grand canonical ensemble is useful in classical statistical mechanics, 
it turns out to be essential in quantum statistical mechanics, as we will see in 
Chapters 26, 27, and 28. Since the basic idea is the same in both classical and quantum 
statistical mechanics, it is convenient to see how the grand canonical ensemble works 
in the classical case, without the added complexity of quantum mechanics. 

20.2 Grand Canonical Probability Distribution 

To describe a system that can exchange energy and particles with a reservoir, we must 
greatly expand our description of a microscopic state. Both the microcanonical and 
canonical ensembles for a system with N particles required only a 6iV-dimensional 
phase space to describe a microscopic state. When the particle number can vary, we 
need a different 6iV-dimensional phase space for each value of N. Since N can vary 
from zero to the total number of particles in the reservoir—which we can take to be 
infinite—this is a significant change. 

To find the probability of a state in this expanded ensemble of an infinite number 
of phase spaces, we return to the basic calculation of the probability distribution in a 
composite system. Assume that the reservoir and the system of interest can exchange 
both energy and particles with each other, but are completely isolated from the rest 
of the universe. The total number of particles is Nt = N + Nfi and the total energy 
is Et = E + Er, where the subscripted R indicates properties of the reservoir. 

The probability of the system of interest having N particles and energy E is given 
by a generalization of eq. (19.10). 


n{E, E, N)nR{ET - E, Vr, Nt - N) 
^ V^, Nt') 


P{E,N) 


(20.3) 


Following the same procedure as in Section 19.3, we take the logarithm of eq. (20.3) 
and expand lnn/j(i?T ~ E, V/j, Nt — N) in powers of E/Et and N/Nt- 

In P{E, N) = In fl{E, V, N) + In ^^(Et - E,Vr,Nt- N) 


— In ^t{Etj Vt, Nt) 

In n{E, E, N) + In Vr, Nt) 



dN Vr, Nt) 

— In Ur{Et,Vt, Nt) 


(20.4) 
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In eq. (20.4) we have neglected terms of higher order in E/Ej- and N/Nt, since the 
reservoir is assumed to be much bigger than the system of interest. 

Recalling that 


Sr = kBlnnii{ET,VR,NT) (20.5) 

we see that we have already found the derivatives of Infl in eq. (8.12). Therefore, 
Pr = l/ksTR is given by 


Pr = -Q^^Ttii}R{ET,VR, Nr) ( 20 . 6 ) 

Similarly, by comparison with eq. (8.30), we see that the chemical potential of the 
reservoir is given by the other partial derivative in eq. (20.4). 

—PrPr = -g^^Ttii}R{ET,VR, Nt) (20.7) 

Since P = Pr and jj, = ^r, and the quantities Er, Nr, Vr do not depend on either 
E or N, we can combine In ^1r{Er, Vr, Nr) and In Q.r{Er, Vr, Nr) into a single value 
denoted by — In 2^. 


In P{E, N) « In n{E, V, N) - PE + P^N - In Z (20.8) 

Exponentiating this equation gives us the grand canonical probability distribution for 
E and N. 


P{E, N) = ^n{E, V, N) exp [-PE + PfiN] (20.9) 

/It 

The normalization condition determines Z, which depends on both T = l/ksP 
and n (and, of course, V). 

^ pOO 

2='^ dEn{E,V,N)exp[-PE +P^iN] (20.10) 

The lower limit in the integral over the energy in eq. (20.10) has been taken to be 
zero, which is the most common value. More generally it should be the lowest allowed 
energy. 

The normalization ‘constant’ Z is called the grand canonical partition function. It 
is constant in the sense that it does not depend on E or N, although it does depend 
on P and /i. 

The grand canonical partition function can also be expressed as a Laplace trans¬ 
form of the canonical partition function. Since eq. (19.18) gave the canonical partition 
function as 


Z{T,V,N) 


dE n{E, V, N) exp{-PE)dE 


( 20 . 11 ) 
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we can rewrite eq. (20.10) as 


C30 

Z(r, V,fi)=Y, Z{T, V, N) exp [/3/ilV] (20.12) 

iV=0 


20.3 Importance of the Grand Canonical Partition Function 

The grand canonical partition function plays much the same role as the canonical par¬ 
tition function. It is directly related to the grand canonical thermodynamic potential 
t/[T, /i]. To see this, consider the value of In P{E, N) at its maximum. The location of 
the maximum, as before, gives the equilibrium values of if = Ef,g = U and N = N^q. 
We can rewrite eq. (20.8) to solve for InZ. 

In Z = In n{Eeg, V, N^q) - PE^g + PfiN.g - In P{E,g, N,g) (20.13) 

We already know that the first three terms on the right of eq. (20.13) are proportional 
to the size of the system. However, P{Eeq, Neq) will be proportional to and 

SO that In P{Eeq, Neq) will only be of order InEeq and IniVeq. For N « 10^°, 
lnP{Eeq,Neq) is Completely negligible, and we can discard it in eq. (20.13). 

InZ = lnn{E,V,N)- PE + PuN (20.14) 

Since we know that the microcanonical entropy is given by 

S = kBlnn (20.15) 

we can rewrite eq. (20.14) as 

InZ = S/kB - PE + PfiN (20.16) 

or 

InZ = -PiE -TS - nN) = -pU[T,fi] (20.17) 

If the system is extensive, Euler’s equation, eq. (13.4), holds, and 

In Z = -PU[T, n] = PPV (20.18) 

This equation gives us a convenient way to calculate the pressure as a function of T 
and as illustrated below for the classical ideal gas in eq. (20.21). 

Eq. (20.18) is directly analogous to eq. (19.48). It provides a way to calculate the 
fundamental relation, and we can immediately invoke everything we learned in Part II 
about how to use thermodynamic potentials to calculate thermodynamic properties. 
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20.4 2{T, V, for the Ideal Gas 

In Section 19.10.1 we derived the partition function for the classical ideal gas, with 
the result given in eq. (19.62). 


Inserting this equation into eq. (20.12) for the grand canonical partition function, we 
find 


Z{T,V,pi)=J2 


1 


Af=0 




{2TTmkBT)^^^^ exp [(3piN] 


N=o \ h 


N 


= exp 




(20.19) 


Taking the logarithm of eq. (20.19) gives us the grand canonical thermodynamic 
potential for the classical ideal gas. 


-PU[T, = {2TTmkBTf'^ = f3PV (20.20) 


The last equality is, of course, true only because the ideal gas is extensive. Note that 
the pressure depends only on /3 and /r, and can be obtained by dividing out the product 
/3V in eq. (20.20). 

P = kBT {2-KmkBTf'‘^ (20.21) 


20.5 Summary of the Most Important Ensembles 

The three ensembles that we have discussed so far are the most important. 

The microcanonical ensemble fixes the energy, volume, and number of particles. 
The formula for the entropy of an isolated system is 

S = kBlnn{E,V,N) (20.22) 

where 

n{E,V,N) = JdqJ dpS{E-H{p,q)) (20.23) 

The origin of these equations is Boltzmann’s definition of the entropy of a composite 
system in terms of the logarithm of the probability distribution. Eq. (20.22) can also 
be written in the form 


n{E, E, N) = exp(5(E, E, N)/kB) 


(20.24) 
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The canonical ensemble describes a system in equilibrium with a thermal 
reservoir. The canonical partition function is given by 

Z = J dEn{E,V,N)exp{-PE) (20.25) 

and is related to the Helmholtz free energy F by 

Z = exp(-/3F) (20.26) 

The grand canonical ensemble describes a system that can exchange both energy 
and particles with a reservoir. The grand canonical partition function is given by 

^ pOO 

Z=^/ dEn{E,V,N)exp[-/3E +PfiN] (20.27) 

and its relationship to U[T,ii] is 

Z = exp(-/3C/[T, /i]) = exp{f3PV) (20.28) 

where the last equality is valid only for extensive systems. 


20.6 Other Classical Ensembles 

For every thermodynamic potential there is a corresponding ensemble in statistical 
mechanics. Every Legendre transform in thermodynamics corresponds to a Laplace 
transform in statistical mechanics. In each case, the logarithm of a partition function 
produces a thermodynamic potential. The only feature that is slightly different is found 
in the grand canonical partition function, for which the Laplace transform takes the 
form of a sum instead of an integral. None of these transforms should present any 
difficulties, since they are all derived using the same principles. 


20.7 Problems 


Problem 20.1 

A non-extensive thermodynamic system 

Consider a classical gas of N weakly interacting atoms of mass to, enclosed in a 
container of volume V and surface area A. 

The interactions between the atoms are so weak that they may be neglected 
(ideal gas approximation). However, the interaction of the atoms with the walls of 
the container may NOT be neglected. Some fraction of the atoms are adsorbed onto 
the walls of the container in equilibrium. Your job will be to calculate the average 
number A' of adsorbed atoms on the walls. 
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A simple model for the atoms adsorbed onto the walls is to treat them as a two- 
dimensional ideal gas. The energy of an adsorbed atom is taken to be 


e{p) = 


M! 

2m 


- So 


where p is the two-dimensional momentum and eo is a known parameter that describes 
the energy of adsorption. 

The entire system is in equilibrium and in thermal contact with a heat reservoir 
at temperature T. 

1. What is the classical partition function of the adsorbed atoms if N' of them are 
adsorbed onto the walls of the container? 

2. What is the chemical potential ps of the atoms adsorbed on the surface? 

3. What is the chemical potential py of the N — N' atoms in the volume of the 
container? 

4. When the atoms in the volume and those adsorbed on the walls are in equilibrium, 
what is the average number of atoms adsorbed as a function of the temperature? 

5. What are the high- and low-temperature limits of the ratio of the number of 
adsorbed atoms to the total number of atoms? 
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Irreversibility 


It would be much more impressive if it flowed the other way. 
Oscar Wilde, on seeing the Niagara Falls 


21.1 What Needs to be Explained? 

The first thing we must establish is the meaning of the term ‘irreversibility’. This is 
not quite as trivial as it might seem. ^ The irreversible behavior I will try to explain 
is that which is observed. Every day we see that time runs in only one direction in 
the real world. If I drop my keys, they fall to the floor and stay there; keys lying on 
the floor do not suddenly jump into my hand. This asymmetry of time has seemed 
to many as being incompatible with the time-reversal invariance of the fundamental 
equations of both classical and quantum physics.^ This is the issue addressed in this 
chapter. 

As with the development of statistical mechanics in general, the explanation of 
irreversibility given here is based on large number of particles in a macroscopic system 
for which we have very limited information about the microscopic state. As in the rest 
of the book, this leads us to describe a macroscopic system on the basis of probability 
theory. 

We will present an explanation of irreversibility using the example of the free 
expansion of a classical ideal gas. The microscopic equations of motion for this 
problem are time-reversal invariant, but the macroscopic behavior will nevertheless 
turn out to be irreversible. Because of the simplicity of the example we will be able 
to carry out every mathematical step exactly, so that the argument can be analyzed 
completely. ^ 


^This chapter is based on an earlier paper by the author; R. H. Swendsen, ‘Explaining Irreversibility’, 
Am. J. Phys., 76, 643-648 (2008). 

^The history of the debate on the origins of irreversibility is fascinating (and continuing), but we will not 
have space to delve into it. There are many books on this topic that the interested student might consult. 

^The mathematical analysis we use was first derived by H. L. Frisch, ‘An approach to equilibrium,’ Phys. 
Rev. 109, 22-29 (1958). 
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21.2 Trivial Form of Irreversibility 

The irreversibility observed in daily life must be distinguished from a trivial form of 
irreversibility that appears only in infinite systems. 

Consider a particle moving in empty space. At some initial time it is observed to 
be located in some finite region of space. Since the particle is moving, it will eventually 
leave this region. If space is infinite, it will not return. This is, technically, irreversible 
behavior, but of a trivial kind. 

This trivial form of irreversibility is real, and occurs in the radiation of light from 
a star. It is quite general in infinite systems, whether classical or quantum. An open 
system also displays this trivial form of irreversibility, since it is really just a piece of 
an infinite system. 

However, we would like to separate this trivial irreversibility from the non-trivial 
irreversibility that we experience every day. Therefore, we will restrict the discussion 
to an isolated, finite system. 

21.3 Boltzmann’s H-Theorem 

The history of the debate on irreversibility has been closely associated with responses 
to Boltzmann’s famous H-theorem, in which he claimed to explain irreversibility. 
Although objections were aimed at Boltzmann’s equations rather than the apparent 
paradox itself, discussing them can help clarify the problem. 

Boltzmann derived an equation for the time-derivative of the distribution of 
atoms in a six-dimensional space of positions and momenta by approximating the 
number of collisions between atoms (Stosszahlansatz). This approximation had the 
effect of replacing the true macroscopic dynamics by a process that included random 
perturbations of the particle trajectories. Within the limits of his approximation, 
Boltzmann showed that a particular quantity that he called ‘H’ could not increase 
with time. 

The fact that Boltzmann used an approximation—and an essential one—in deriv¬ 
ing his result meant that his derivation could not be regarded as a proof. The 
arguments began soon after his publication appeared, and have continued to the 
present day. 

21.4 Loschmidt's Umkehreinwand 

The first objection came from Boltzmann’s friend, Johann Josef Loschmidt (Austrian 
physicist, 1821-1895), who noted that if all momenta in an isolated system were 
reversed, the system would retrace its trajectory. If Boltzmann’s H-function had been 
decreasing at the moment of reversal, its value after time-reversal must increase. 
Loschmidt argued on this basis that Boltzmann’s conclusions could not be correct, 
and his argument is usually referred to by the German term Umkehreinwand. 

Loschmidt’s argument is very close to the central paradox. If every microscopic 
state that approaches equilibrium corresponds to a time-reversed state that moves 
away from equilibrium, should they not be equally probable? 
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21.5 Zermelo’s Wiederkehreinwand 

Ernst Zermelo (German mathematician, 1871-1953) was a prominent mathematician, 
who raised a different objection. He cited a recently derived theorem by Jules Henri 
Poincare (French mathematician and physicist, 1854-1912) that proved that any 
isolated classical system must exhibit quasi-periodic behavior; that is, the system must 
return repeatedly to points in phase space that are arbitrarily close to its starting 
point. Zermelo claimed that Poincare’s theorem is incompatible with Boltzmann’s 
H-theorem, which predicts that the system will never leave equilibrium. Zermelo’s 
argument is referred to by the German term Wiederkehreinwand. 


21.6 Free Expansion of a Classical Ideal Gas 

In this section we will present an exact analysis of a simple model that exhibits 
irreversible behavior. Since the model is governed by time-reversal-invariant equations 
of motion, this will serve as a demonstration of the compatibility of irreversible 
macroscopic behavior and time-reversal invariance. 

Gonsider an ideal gas of N particles in a volume V. Isolate the gas from the rest 
of the universe and assume that the walls of the box are perfectly reflecting. Initially, 
the gas is confined to a smaller volume Vo by an inner wall. At time t = 0, the inner 
wall is removed. 

For convenience, let the box be rectangular and align the coordinate axes with 
its sides. The inner wall that initially confines the gas to a subvolume Vo is assumed 
to be perpendicular to the x-direction and located at x = Lo- The length of the box 
in the x-direction is L. The dependence of the probability distribution on the y- and 
z-coordinates does not change with time, so that we can treat it as a one-dimensional 
problem. 

At time t = 0, the confining wall at Lq is removed and the particles are free to move 
throughout the box. Following Frisch’s 1958 paper, we will eliminate the difficulties 
involved in describing collisions with the walls at x = 0 and x = L hy mapping the 
problem onto a box of length 2L with periodic boundary conditions. When a particle 
bounces off a wall in the original system, it corresponds to a particle passing between 
the positive and negative sides of the box without change of momentum. The mapping 
from the periodic system to the original system is then simply Xj \xj\. 

The key assumption is that the initial microscopic state must be described by 
a probability distribution. Assume that before the inner wall is removed, the initial 
positions of the particles are uniformly distributed in Vo, and the momenta have 
a probability distribution ho{p). This probability distribution is assumed to be time- 
reversal invariant, ho (—Pj) = ho (pj), so that there is no time-asymmetry in the initial 
conditions. Since we are assuming that the positions and momenta of different particles 
are initially independent, we can write the total probability distribution as 


N 


N 


fN{{xj,Pj\j = l...N},t = 0) = Y[f{xj,pj,t = 0) = Y[goixj)ho{pj) (21.1) 


i=i 


i=i 
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Fig. 21.1 The initial probability distribution at t = 0. 


where 


9o (xj) 


f 1/Lo -Lo < X < Lq 
\0 |a;| > Lq 


( 21 . 2 ) 


This initial probability distribution for the periodic system is shown schematically in 
Fig. 21.1. 

Since the particle probabilities are independent, we can restrict our attention to 
the distribution function of a single particle. The periodic boundary conditions then 
allow us to Fourier transform the initial conditions to obtain 


OO 

g{x) = + cos 

n—1 


(21.3) 


where no sine terms enter because of the symmetry of the initial conditions in the 
expanded representation with period 2L. 

Using the standard procedure of multiplying by cos(7rn'x/T) and integrating to 
obtain the coefficients in eq.(21.3), we find 



For n' = 0, this gives us 



(21.5) 
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and for n' > 1 


9n' — 


2 

-sin 

n'lrLo 



The time development for a given momentum is now simple. 


( 21 . 6 ) 


/ t) = f (^x- t,p, o) 


= flo (p) 




(21,7) 


Fig. 21.2 shows the probability distribution at a time t > 0. The shaded areas, 
representing non-zero probabilities, tilt to the right. As time goes on, the shaded 
areas become increasingly flatter and closer together. 

To And an explicit solution for a special case, assume that the initial probability 
distribution for the momentum is given by the Maxwell-Boltzmann distribution. 


ho (p) 


P 

271771 


1/2 

exp 



( 21 . 8 ) 


where T is the temperature, and /3 = l/ksT. The assumption of a Maxwell-Boltzmann 
distribution is not essential, but it provides an explicit example of how the approach 
to equilibrium comes about. 



Fig. 21.2 The probability distribution for t > 0. 
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Inserting eq. (21.8) in eq. (21.7), we can integrate over the momenta to calculate 
the local probability density as a function of time. 


/ OO 

/(a;, t) dp 

-OO 

/ OO 

ho (p) L~^dp 

-OO 


+ 


/* OO 


The first integral is trivial, since hg {p) is normalized. 

/ OO 

ho (p) dp = l 

-OO 


( 21 . 10 ) 


The integrals in the sum are rather tedious, but not particularly difficult when you 
use the identity cos0 = | (e*® + to write them in the form 


f 


exp 


p^ nn.p 

-13— ± -j-i—t 

Zm L m 


dp 


( 21 . 11 ) 


and complete the square in the exponent. The result is 

9 1 

g {x, t) = ^ - sin cos (^a^) exp [-Xlf] (21.12) 


where 


n=l 


A" = 


2mL^P 2mL'^ 


keT 


(21.13) 


Note that since the temperature is related to the one-dimensional root-mean-square 
velocity by 


l/2\ ^ / 2 \ ^2 


the coefficients A„ can be written as 


2 ^ 2 
'^n ^rms 


(21.14) 


(21.15) 


Eq. (21.12) can also be expressed in terms of the characteristic time for a particle 
traveling with the speed Vrms to cross the box. 


L 


(21.16) 
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In terms of r, the time-dependent local probability density becomes 


g{x,t) = L ^ 1 


L 






Since 




(21.17) 

(21.18) 


for all X, the sum converges rapidly for t > 0. At t = r, the n-th term in the sum is 
less than (;^) exp — ■, and even the leading non-constant term has a relative 

size of less than 0.005 ■ The rapidity of convergence is striking. The inclusion of 

interactions in the model would actually slow down the approach to equilibrium. 

It is also possible to calculate the energy density as a function of position and time 
by including a factor of p^/2m in the integral over the momentum. 


U{x,t)=J f {x,p,t) dp 

+|;s” Ij'- (p) (h) (t ("- (£)')) ‘‘p 


The integrals are even more tedious than those in eq. (21.9), but again not particularly 
difficult. 


U (x, t) 


1 

2L^ 



L'^Lommrfd'^ ) 





( 21 . 20 ) 


After the internal wall is removed, the energy distribution is not uniform and not 
proportional to the particle density, since the faster particles move more rapidly into 
the region that was originally vacuum. However, the energy density converges rapidly 
to the expected constant, 1/2L/3 = hfcsT/L, as t ^ oo. 


21.7 Zermelo’s Wiederkehreinwand Revisited 

We have now established that that the particle density and energy density both go to 
equilibrium. This leaves the question of reconciling the approach to equilibrium with 
Poincare recurrences. Fortunately, it is easy to see how quasi-periodic behavior can 
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arise in the ideal gas model, without contradicting the approach to equilibrium. To do 
this, it helps to have an intuitive picture of how Poincare cycles occur. 

First consider two particles with speeds Vi and V 2 - They will each return to their 
original states with periods ti = 2Ljvi and T 2 = 2 L/v 2 ^ respectively. In general, the 
ratio Ti/r 2 will be irrational, but it can be approximated to arbitrary accuracy by 
a rational number, ri/T 2 « niln 2 , where rii and n 2 are sufficiently large integers. 
Therefore, after a time ri 2 = ri 2 'ri « riir 2 , both particles will return to positions and 
velocities arbitrarily close their initial states . 

Now add a third particle with speed ^3 and period T 3 = 2 L/V 3 . A rational 
approximation Ti_ 2 /'n 3 ~ will give us an approximate recurrence after a time 

''"1,2,3 = ni^2T3 ~ n3Ti_2 ~ n3n2Ti. Since 712 and 113 will usually be large numbers, ti_2,3 
will usually be a long time. 

If we repeat this procedure for 10^^ particles, we will arrive (with probability one) 
at a recurrence time that would be enormous even in comparison to the age of the 
universe. It might naturally be said that we are not interested in such extremely long 
times, but it is interesting to note that eqs. (21.17) and (21.20) do not exhibit the 
quasi-periodic behavior that Poincare recurrence might seem to require. 

The resolution of the apparent contradiction lies in our lack of knowledge of the 
exact initial velocities and the extreme sensitivity of the Poincare recurrence time to 
tiny changes in initial velocities. Even with far more detailed information than we are 
ever going to obtain in a real experiment, we would not be able predict a Poincare 
recurrence with an uncertainty of less than many ages of the universe. 

Poincare recurrences are included in the exact solution for the free expansion of an 
ideal gas. However, since we cannot predict them, they appear as extremely rare large 
fluctuations. Since such large fluctuations are always possible in equilibrium, there is 
no contradiction between observed irreversible behavior and Poincare recurrences. 


21.8 Loschmidt’s Umkehreinwand Revisited 

Loschmidt’s Umkehreinwand was directed against Boltzmann’s equation, which was 
not time-reversal invariant. However, the exact solution to the free expansion of an 
ideal gas retains the time-reversal properties of the microscopic equations. If we reverse 
all velocities in the model some time after the inner wall is removed, the particles return 
to their original positions. 

Under normal conditions, the reversal of molecular velocities is experimentally 
difficult, to say the least. However, the reversal of spin precession can be accomplished 
and is fundamental to magnetic resonance imaging. It is known as the spin-echo effect. 
After an initial magnetic pulse aligns the spins in a sample, interactions between the 
spins and inhomogeneities in the sample lead to decoherence and a decay of this 
magnetization. If no further action is taken, the signal will not return. However, if a 
second magnetic pulse is used to rotate the spins by 180° at a time t after the first 
pulse, it effectively reverses the direction of precession of the spins. The spins realign 
(or ‘refocus’) after a total time 2 t after the initial pulse, and the magnetization appears 
again. 
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The reversal of the time-development of an ‘irreversible’ process is therefore not a 
flaw in the free-expansion model, but a reflection of the reality that such experiments 
can be carried out. 

21.9 What is‘Equilibrium’? 

An interesting feature of the exact solution to the free-expansion model can be seen in 
Fig. 21.2. Even though the shaded regions, indicating non-zero probabilities, become 
progressively thinner and closer together, the local probability density at any point 
along the trajectory of the system remains constant. This is consistent with Liouville’s 
theorem that the total time derivative of the probability distribution vanishes.^ 

This property of all isolated Hamiltonian systems has caused difficulties for those 
who would like to define equilibrium in terms of a smooth distribution in phase space. 
Although we assumed that our model experiment started with a canonical probability 
distribution, it will certainly never evolve to one. 

I believe that the difficulty is a confusion about the direction of inference. It 
has been amply demonstrated that a canonical distribution accurately describes an 
equilibrium state. However, that does not imply that a system in equilibrium can only 
be described by a canonical distribution. The probability distribution for our model 
at long times will give the same macroscopic predictions as the canonical distribution, 
even though the two distributions are different. 

21.10 Entropy 

Liouville’s theorem has also caused difficulties for the traditional textbook definition 
of the entropy as the logarithm of a volume in phase space. The theorem requires 
that this volume remain constant, so the traditional expression for the entropy cannot 
increase in an isolated system. 

This seems to violate the Second Law, but it is correct—in a certain sense. The 
traditional definition of entropy is related to the total information we have about the 
system—not the thermodynamic information about the current and future behavior of 
the system. The information that the system was initially confined to a smaller volume 
is contained in the layered structure of the probability density shown in Fig. 21.2. 
Since that information does not change, the traditional entropy does not change. The 
apparent violation of the Second Law arises because the traditional entropy does not 
correspond to the thermodynamic entropy. 

If we use Boltzmann’s definition of the thermodynamic entropy in terms of the 
probability of a macroscopic state, we obtain an expression that increases with time 
as expected for the thermodynamic entropy. 

The specific form of the entropy depends on what we are measuring in the 
experiment. Since we are interested in the time development of the system as it 
approaches a uniform state, it would be reasonable to observe its properties as a 
function of position. Let us divide the system into M subsystems, each with length 

^See Section 19.5 for a derivation of the Liouville theorem. 
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AL = L/M, and measure the energy and density in each subsystem. For convenience, 
assume that the first m subsystems were inside the inner wall before the free expansion 
began, so that Lq = mL/M. 

If M is large, we can assume that the energy and density are uniform across a 
subsystem. The number of particles in the j-th subsystem, Nj{t), is given by AL 
times the expression in eq. (21.12), and the energy in the j-th subsystem, Ej{t), is 
given by AL times the expression in eq.(21.20) 

If the subsystems are large enough that l/y^Nj{t) is much smaller than the relative 
error in experimental measurements, they can be regarded as macroscopic systems. 
Their individual entropies might then be well approximated by the equilibrium entropy 
of a one-dimensional, classical ideal gas. 


S{E,{t),L/M,N,{t)) 


Nj{t)kBT 


In 


L/M 


+ iln 


Ej{t) 

N,{t) 


+ X 


( 21 . 21 ) 


where X is the usual constant. The total time-dependent entropy of the whole system 
is then given by 


M 


S{{E,{t)},L,N,{t),M) = Y,S {E, it ), L/M, N, {t)) 


( 21 . 22 ) 


This expression has the properties that at t = 0, it takes on the value of the initial 
entropy before the inner wall was removed. 


S{{E,{0)},L,N,{0),M) = S{E,Lo,N) (21.23) 


and as t ^ oo, it goes rapidly to the equilibrium entropy of the full system. 


lim S{{Ej{t)},L,Nj{t),M) = S{E,L,N) (21.24) 

t —>00 

These two properties are independent of the number M of subsystems that are 
observed. 


21.11 Interacting Particles 

The explanation of irreversible phenomena given here is, of course, not complete. The 
apparent conflict between microscopic and macroscopic laws has been resolved, but 
we have ignored the effects of interactions. However, now that it is clear that there 
is no real conflict between time-reversal-invariant microscopic laws and macroscopic 
irreversibility, it should be sufficient to demonstrate the equilibration of the momentum 
distribution by molecular dynamics computer simulations that are generalizations of 
those carried out in the assignments in Chapter 19. 
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Quantum Ensembles 


/ think I can safely say that nobody understands quantum mechanics. 
Richard Feynman, in The Character of Physieal Law 


In Part IV of this book we will extend the application of statistical ideas to quantum 
mechanical systems. Most of the results we will derive have counterparts in classical 
statistical mechanics, but some important features will be quite new. 

The differences between classical and quantum statistical mechanics are all based 
on the differing concepts of a microscopic ‘state’. While the classical microscopic state 
(specified by a point in phase space) determines the exact position and momentum 
of every particle, the quantum mechanical state determines neither; quantum states 
can only provide probability distributions for observable quantities. This feature has 
the important consequence that quantum statistical mechanics involves two different 
kinds of probability, while classical statistical mechanics involves only one. 

Classical and quantum statistical mechanics both require the assignment of proba¬ 
bilities to the possible microscopic states of a system. The calculation of all quantities 
involves averages over such probability distributions in both theories. 

Quantum statistical mechanics further requires averages over the probability dis¬ 
tributions that are obtained from each individual microscopic state. 

We will begin this chapter by recalling the basic equations of quantum mechanics, 
assuming that the reader is already familiar with the material. If any part of the 
discussion seems mysterious, it would be advisable to consult a textbook on quantum 
mechanics. 

After a review of the basic ideas we will discuss the special features that distinguish 
quantum statistical mechanics from the classical theory. 

Subsequent chapters will develop the theory of quantum statistical mechanics and 
apply it to models that demonstrate the significant differences between the predictions 
of quantum and classical statistical mechanics. We will show that even simple harmonic 
oscillators and ideal gases have dramatically different properties in quantum and 
classical theories. Since the real world ultimately obeys quantum mechanics, quantum 
statistical mechanics is essential to obtain agreement with experimental results. 



248 Quantum Ensembles 


22.1 Basic Quantum Mechanics 

The most fundamental characteristic of quantum mechanics is that the microscopic 
state of a system is described by a wave function, rather than a point in phase space. 
For a simple, one-particle system, the wave function can be represented by a complex 
function of space and time, (r, t), which contains all available information about the 
particle. 

Even if the wave function is known exactly, the result of a measurement of the 
position of the particle can only be predicted in terms of probabilities. For example, 
the absolute value of the square of the wave function, {tp (r, t) p, gives the probability 
density for the result of a measurement of the position of a particle. (After the 
measurement, of course, the wave function of the particle would change.) Only the 
average position of the particle and the various moments of its probability distribution 
can be calculated from the wave function. 

The momentum, p, is represented by a vector operator rather than a real vector. 



( 22 . 1 ) 


As is the case for the position of the particle, the momentum does not have a fixed 
value. The expectation value of the momentum is given by an integral over the wave 
function. This can be denoted as 



( 22 . 2 ) 


In eq. (22.2), tp* denotes the complex conjugate of the function tp, and we have 
introduced the ‘bra’, {tp\ = tp* and ‘kef, \tp) = tp{f,t) notation. 

The kinetic and potential energy are represented by a Hamiltonian. 



In this equation. 



(22.4) 


The wave function must satisfy the Schrodinger equation. 



(22.5) 


22.2 Energy Eigenstates 

There are certain special quantum states that can be written as a product of a function 
of position times a function of time. 


tp{r,t) = tp{r)f{t) 


(22.6) 
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For these states, the Schrodinger equation can be written in a form that separates the 
functions of time and space. 


Since the left side of eq. (22.7) depends only on time and the right side depends 
only on space, they must both be equal to a constant, which we will denote as E. For 
these special states, the Schrodinger equation separates into two equations: 


and 

H (^r, —ihV^ tp (r) = Etp (r) 
Since the expectation value of E[ is just 

{H)=E 


( 22 . 8 ) 


(22.9) 


( 22 . 10 ) 


we can identify E as the energy of the state. 

Eq. (22.8) can be easily solved as a function of time. 

/(t) = exp (22.11) 


In general, eq. (22.9) only has solutions for particular values of E, which are the 
only observable values of the energy. These special values are known as energy eigen¬ 
values, and the corresponding wave functions are known as eigenfunctions. Eq. (22.9) is 
called an eigenvalue equation. The eigenfunctions and eigenvalues can be identified by 
a quantum ‘number’ n, which is really a set of d numbers that describe the eigenstate 
for a wave function in a d-dimensional system. 

Including the quantum number explicitly, the eigenvalue equation can be written 
as 


Htpn (fO = Entjjn {r) 

(22.12) 

H\n) = En\n) 

(22.13) 


where we have suppressed the explicit dependence on r in the second form of the 
eigenvalue equation. E^ is called the energy eigenvalue. It is possible for distinct 
eigenstates to have the same energy eigenvalue, in which case the states are called 
‘degenerate’. 

The time-dependence of an eigenstate therefore has the form 


Ipn {r, t) = ipn (^ exp 


(22.14) 
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Since the expectation value of any operator is constant in time for eigenstates, they 
are also called ‘stationary states’. 

It is standard procedure to normalize the eigenfunctions. 

(n|n) = = l (22.15) 

In this equation, we have again used the ‘bra-ket’ notation (tjtn (r, t) = K))- 

It is easily proved that for En ^ Em, (n|m) = 0. It is also straightforward to chose 
eigenfunctions in such a way that for n ^ m, (n|m) = 0, even when En = Em- These 
properties can be summarized using the Kronecker delta. 

(n|m) = 5n,m (22.16) 


22.2.1 Expansion of a General Wave Function 

A very important theorem is that any wave function can be expanded in the set of all 
eigenfunctions. 


^ (f) = ^ Cniltn (22.17) 

n 

This can also be expressed more compactly using the ket notation. 

\^P)=Y,Cn\n) (22.18) 

n 

The coefficients c„ are complex numbers that can be calculated from the wave 
functions. 


= ^Cn{m\n) = 




dm.m. — 


-'n'-'m.n 


- Cr) 


(22.19) 


22.2.2 Magnitudes of Coefficients 

Since wave functions are assumed to be normalized, there is a sum rule for the c^s. 

1 = (-01-0) = ^ ^ CnCm{i>n\i’m) = X! |CnP (22.20) 

n m n m n 

The expectation value of the energy can also be expressed in terms of the coeffi¬ 
cients {c„} and the energy eigenvalues. 

('0|i7|0) = X! X! CmC„(m|i7|n) (22.21) 

m n 

= '^^CmCn{m\En\n) 
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= ^ ^ EnC'^CnSm, n 

m n 

= y^^nlCnP 

n 

Since the sum of |c„p is 1, and the expectation value of the energy is given by the 
weighted sum in eq. (22.21), it is natural to interpret |c„p as the probability that a 
measurement of the energy will put the system into the eigenstate |n). 

It is important to note that if a system is in a general state, \tjj), given by eq. (22.18), 
it is not in an eigenstate unless |c„| = 1 for some value of n. |c„p is the probability 
that the system would be found in the state \n) only after a measurement has been 
made that put the system into an eigenstate. Without such a measurement, the 
probability that the system is in an eigenstate is zero. 


22.2.3 Phases of Coefficients 

Since the time-dependence of the eigenfunctions is known, the time-dependence of an 
arbitrary wave function can be written in terms of the expansion in eigenfunctions, 

4 i{r,t) ='^CneyL^(-i^t\iin{r) (22.22) 

n ^ ^ 

or more compactly, 

H, t) = '^Cn exp f j l’^) (22.23) 

Eqs. (22.22) or (22.23) show that the time development of an arbitrary quantum 
state can be described by the changing phases of the expansion coefficients. They 
also show that since eigenstates with different energies have phases that change at 
different rates, the relative phases between any two states, given by (£!„ — Em)t/h, 
sweeps uniformly through all angles. This property suggests that a model probability 
distribution in equilibrium should be uniform in the phases. Indeed, if a probability 
distribution is not uniform in the phases, it will not be time-independent. 

22.3 Many-Body Systems 

Quantum systems that contain many particles (‘many-body systems’ in common 
terminology) are also described by a single wave function, but one that is a function of 
the coordinates of every particle in the system: '0 ({f) |j = 1,..., N}). The Hamiltonian 
naturally depends on all the coordinates and all the gradients. 


(22.24) 
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We can again find eigenstates of the Hamiltonian (at least in principle), but the 
quantum ‘number’ n is now a set of 3N numbers that are needed to describe the 
many-body wave function. 

A general many-body wave function can also be expanded as a linear combination 
of eigenstates, as in eq. (22.18), and the time-development of the many-body wave 
function can still be represented by eq. (22.23), with the appropriate interpretation of 
n as the set of many-body quantum numbers. 

As in the single-particle case, the square of the absolute value of a coefficient in the 
expansion of a many-body wave function, |c„p, can be interpreted as the probability 
that a measurement that put the system in an eigenstate would put it in eigenstate 
n. However, such a measurement is never carried out on a macroscopic system— 
which is never in an energy eigenstate! 


22.4 Two Types of Probability 

In Part I of this book we introduced the use of probabilities to describe our ignorance 
of the exact microscopic state of a many-body system. Switching from classical to 
quantum mechanics, we find a corresponding ignorance of the exact microscopic 
wave function. Again we need to construct a model probability distribution for the 
microscopic states to describe the behavior of a macroscopic system. 

However, as we have seen above, the properties of a quantum system are still 
only given by probability distributions even if we know the wave function. Therefore, 
in quantum statistical mechanics we need to deal with two kinds of probability 
distributions: one for the microscopic states, and a second for the observable properties. 

22.4.1 Model Probabilities for Quantum Systems 

We will denote a model probability distribution for the many-body wave functions as 
P^. The calculation of the expectation value (average value) of any operator A must 
include averages over probability distribution of the wave functions and the probability 
distribution of each quantum state. 

(A)= [ P^{tP\A\^} (22.25) 

Jip 

We have written the average over P^ as an integral because there is a continuum of 
wave functions. 

We can write eq. (22.25) in a more convenient form by expressing the wave function 
in terms of an expansion in eigenfunctions, as in eq. (22.18). Since the expansion 
coefficients completely specify the wave function, we can express the probability 
distribution in terms of the coefficients. 


Pip = P{Cr,} 


(22.26) 
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The average of A then becomes 




n m 


(22.27) 


The integral in eq. (22.27) is over all values of all coefficients, subject to the normal¬ 


ization condition in eq. (22.20). 

Since the phases play an important role in specifying the time dependence in 
quantum mechanics, it will be useful to modify eq. (22.27) to exhibit them explicitly. 
If we write the coefficients as 


Cn = \Cn\exp{i4>n) 


(22.28) 


we can rewrite eq. (22.27) as 



(22.29) 


Since we have introduced the phases explicitly into eq. (22.29), we have also separated 
the integrals over the coefficients into integrals over their magnitudes and their phases. 

22.4.2 Phase Symmetry in Equilibrium 

Eq. (22.29) gives a formal expression for the macroscopic average of any property in 
any macroscopic system. However, since we are primarily interested in macroscopic 
equilibrium states, we can greatly simplify the problem by making an assumption 
about the probability distribution of the phases. 

As the wave function in eq. (22.23) develops in time, the relative phases between 
any two eigenstates (with different energy eigenvalues) sweep out all angles. Since the 
macroscopic equilibrium state is time-independent, it would seem reasonable to assume 
that the equilibrium probability distribution of phase angles should be uniform and 
the phases independent. With this assumption, P{c„} = -P{|c„|}j ^nd we can integrate 
over the phase angles when n yf m 



(22.30) 


or n = m 



(22.31) 


Inserting eqs. (22.30) and (22.31) into eq. (22.29) we find 
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or 


m> = E 


/ P{\Cn\}\^n\ 

H\Cr.\} 


i\A\'i 


(22.33) 


Eq. (22.33) has a particularly interesting form. If we define a set of values {Pn}, 
where 


P„ = 27r 


I 


Pi 


{|cj} 


{|c„|}l 


(22.34) 


we can rewrite the equation for (^) in a very useful form. 


M)=i; Pn{n\A\n) (22.35) 

n 

The only quantum averages that appear in eq. (22.35) are of the form {n\A\n), 
so every term is an average over a single quantum eigenstate. For the purposes of 
calculating the macroscopic properties of a macroscopic system in equilibrium, each 
eigenstate contributes separately. 

The P„s have two useful properties. First, since P^ is a probability distribution, 
P^ >Q for all ’ll). From eq. (22.34), it follows that Pn > 0 for all n. Next, since 
the average of a constant must be the value of the constant, eq. (22.35) gives us a 
normalization condition on the P„s. 

1 = = = (22.36) 

n n 

This equation also implies that P„ < 1. 

Since 0 < P„ < 1 for all n, and from eq. (22.36) the sum of all P„s is unity, the 
P„’s fairly beg to be regarded as probabilities. The question is: what are they 
probabilities of? P„ can be interpreted as the probability that a measurement of 
the eigenstate of a system will result in the system being in eigenstate \n). However, 
it must be recognized that such a measurement is never made on a macroscopic 
system. Nevertheless, calculations using eq. (22.35) are carried out formally exactly 
as if the P„s were probabilities of something physically relevant. There is no real 
harm in referring to P„ as the ‘probability of being in eigenstate |n)’ (which is often 
found in textbooks), as long as you are aware of its true meaning. A more complete 
description of why this is a useful—if not entirely correct—way of thinking about 
quantum statistical mechanics is presented in Sections 22.5 and 22.6. 


22.5 The Density Matrix 

A very common representation of averages in a quantum ensemble is provided by 
the ‘density matrix’, the operator of which is defined by a sum or integral over the 
microscopic states in an ensemble. 
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p= f (22.37) 

J Ip 

The density matrix pn,m would be found by taking the matrix element between the 
states \m) and \n). 


Pn,m = [ P,jj{m\'tlj){tj}\n) (22.38) 

J Ip 

The density matrix is useful because it incorporates the information needed to 
calculate ensemble averages. In particular, the ensemble average of an operator A is 
given by 


(^) = Tv [pA] 


(22.39) 


where Tr indicates the trace of the matrix. To prove eq. (22.39), simply evaluate the 
trace on the right-hand side of the equation. 


Tr[pA]=^ f P^{n\tlj){'ip\A\n) 

n 

n 

= f '^'^'^P{c„}Cec^{n\e){m\A\n) 

{^n} n i m 

= [ J2J2^{^A^niCr,{m\A\n) = {A) 


(22.40) 


The last equality is found by comparison with eq. (22.27). 

22.6 The Uniqueness of the Ensemble 

A peculiar feature of the density matrix is that although eq. (22.40) shows that it 
contains enough information to calculate any ensemble average, it does not uniquely 
specify the quantum ensemble. This can be seen from a simple example that compares 
the following two different quantum mechanical ensembles for a simple harmonic 
oscillator. 

1. Ensemble 1: The SHO is either in the ground state |0) with probability 1/2, or it 
is in the first excited state |1) with probability 1/2. In either case, the system is 
in an eigenstate of the Hamiltonian. The density matrix operator corresponding 
to this ensemble is then 


Pi = ^(|0)(0| + |l)(l|) 


(22.41) 
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2. Ensemble 2: The system is in a state of the form 


!</>) = ^(|0) + e^1l)) 


(22.42) 


with the values of (j) being uniformly distributed between 0 and 27r. No member 


of the ensemble is an eigenstate of the Hamiltonian The density matrix operator 
corresponding to this ensemble is then 



(22.43) 


This shows that the same density matrix operator describes both an ensemble that 
contains no eigenstates and an ensemble that contains only eigenstates. If we combine 
this with the result in eq. (22.40), we see that the expectation value of any operator 
is exactly the same for different ensembles as long as the density matrix operators are 
the same—even if the two ensembles do not have a single quantum state in common! 

It is usual to express the density matrix operator in terms of the eigenstates of 
a system, which gives the impression that the ensemble also consists entirely of 
eigenstates. Even though we know that this is not the case, the demonstration above 
shows that all predictions based on the (erroneous) assumption that a macroscopic 
system is in an eigenstate will be consistent with experiment. 


22.7 The Quantum Microcanonical Ensemble 

Now that we have reduced the problem of calculating averages in equilibrium quantum 
statistical mechanics to averages over individual eigenstates, we need only evaluate the 
set of ‘probabilities’, {T’n}, where n is the quantum number (or numbers) indexing the 
eigenstates. In principle this is straightforward, and we might try to follow the same 
procedure as in classical statistical mechanics. Consider a composite system, isolated 
from the rest of the universe, with fixed total energy Et = Ea + Eb, assume all 
states are equally ‘probable’ (subject to the constraints), and calculate the probability 
distributions for observable quantities. 

Unfortunately, we run into a technical difficulty that prevents us from carrying out 
this procedure for anything other than one particularly simple model. 
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If we represent each subsystem by its set of eigenfunctions and energy eigenvalues, 
the total energy of the composite system must be the sum of an energy eigenvalue from 
each subsystem, E = En^A + Em,B- To calculate the probabilities for distributing the 
total energy among the two subsystems, we have to be able to take some amount of 
energy from subsystem A and transfer it to subsystem B. Unfortunately, the energy 
levels in a general macroscopic quantum system are not distributed in a regular fashion; 
if we change the energy in subsystem A to E^’^a = -E’n.A + there will generally not 
be an eigenstate in subsystem B with an energy Em,B — AE. Even though the energy 
eigenvalues in a macroscopic system are very closely spaced, we will not generally be 
able to transfer energy between the subsystem and maintain a constant total energy. 

It is important to recognize that this is a technical problem that has arisen because 
we are treating each subsystem as a separate quantum system with its own set of 
eigenfunctions and energy eigenvalues. It is not a fundamental inconsistency. 

First of all, because macroscopic systems are never in an eigenstate, they do not 
have precisely specified energy, and the typical spread in energy in typical states is far 
greater than the tiny gap between energy levels. Beyond that, the interactions between 
the two subsystems that are necessary to transfer energy between them will result in 
a single set of eigenvalues for the full composite system. 

The fact that the difficulty is purely technical is comforting, but we must still find 
a way to do calculations. The most useful method is to make one of the subsystems 
extremely large and treat it as a thermal reservoir. As a quantum system becomes 
larger, its energy levels move closer together; in the limit of an infinite system, the 
energy spectrum becomes a continuum, and it is always possible to find a state with 
any desired energy. For this reason, we will abandon the microcanonical ensemble and 
turn to the canonical ensemble for most calculations in quantum statistical mechanics. 
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I do not like it, and I am sorry I ever had anything to do with it. 
Erwin Schrodinger, Austrian physicist (1887-1961), Nobel Prize 1933, 
speaking about quantum mechanics. 


The quantum canonical ensemble, like its classical counterpart, which was discussed 
in Chapter 19, describes the behavior of a system in contact with a thermal reservoir. 
As in the classical case, the detailed nature of the reservoir is not important; it must 
only be big. This is partly to allow us to expand the entropy in powers of the ratio 
of the size of the system to that of the reservoir, and also to allow us to treat the 
eigenvalue spectrum of the reservoir as continuous. 


An important thing to notice about quantum statistical mechanics as you go 
through the following chapters is that the same ideas and equations keep showing 
up to solve different problems. In particular, the solution to the quantum simple 
harmonic oscillator (QSHO) turns out to be the key to understanding crystal 
vibrations, black-body radiation, and Bose-Einstein statistics. The basic calculation 
for the QSHO is discussed in Section 23.10; if you understand it completely, the 
topics in Chapters 24 through 27 will be much easier. The other kind of calculation 
that will appear repeatedly is one in which there are only a small number of quantum 
energy levels. Indeed, in the most important case there are only two energy levels. 
This might seem like a very limited example, but it provides the essential equations 
for Chapters 26, 28, and 30. 


23.1 Derivation of the QM Canonical Ensemble 

The derivation of the quantum canonical ensemble follows the same pattern as the 
derivation of the classical canonical ensemble in Section 19.3. 

Let an eigenstate |n) for the system of interest have energy and the reservoir 
have energy Er, so that the total energy is Et = En + Er- Let ^r{Er) = ^r{Et — 
En) be the degeneracy (number of eigenstates) of the energy level Er in the reservoir. 
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The ‘probability’ of an eigenstate with energy En in the system of interest is then 
given by: 


^r{Et — En) 
dx{Ex) 


(23.1) 


As we did in Section 19.3, take the logarithm of both sides and use the fact that 
Et ^ En to expand lnCtfi{ET — En) in powers of En/Ex- 


In Pn = In ^r(Et - En) - In flriEr) 

« lnClR{ET) — En „ Inflfl(ifr) — lnClT{ET) + • • ■ (23.2) 

Ull/'J' 

The approximation becomes exact in the limit of an infinite reservoir. In practice, the 
error due to using finite reservoirs is so much smaller than the errors in measurement, 
that we will ignore the higher-order terms indicated by ‘-I- • • • ’ and use an equal sign 
for simplicity. 

In analogy to the classical derivation in Section 19.3, we identify 

so that we can write eq.(23.2) as 

lnP„ =-lnZ-/3A„ (23.4) 


or 

P„ = ^ exp(-/3P„) (23.5) 

In eqs. (23.4) and (23.5), we have introduced the normalization factor Z = 
In fl/^(if 7 ^) — lnH 7 ^(p 7 "). 

The normalization factor Z is known as the quantum mechanical partition function 
(in analogy to the classical partition function) and can be evaluated by summation 
over all eigenstates. 

Z = ^exp(-/lP„) (23.6) 

n 

Note that the quantum Boltzmann factor exp(—/JP^) plays the same role as the 
classical Boltzmann factor in eq. (19.17). The quantum mechanical partition function 
plays the same role as the classical partition function. 

Eq. (23.6) is often used as a starting point in textbooks on quantum statistical 
mechanics because of its simplicity. Unfortunately, its validity and significance are 
not completely obvious without an explanation. It is a good starting point for 
many calculations in quantum statistical mechanics. However, a problem arises if 
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we consider a quantum theory of distinguishable particles. A common procedure is 
to ignore this possibility, but since a quantum theory of colloidal particles (which 
are distinguishable) should make sense, we will include this case in a later chapter. It 
should not come as a surprise that a factor of 1/iV! will play a role in the discussion, 
and eq. (23.6) will have to be modified to include the factor of 1/7V! explicitly. 


It is often convenient to reexpress eq. (23.6) in terms of the energy levels. In that 
case, we must include the degeneracy II(£) for each energy level £. 

Z = Y^ n{(.) exp(-/3A|) (23.7) 

i 


It is essential to remember that in eq. (23.6) the index n runs over the quantum 
numbers for the eigenstates of the system—not the energy levels—and that you 
need to include the degeneracy as in eq. (23.7) if you are summing over energy 
levels. 


23.2 Thermal Averages and the Average Energy 

Combining eq. (23.6) with eq. (22.35), we find a very useful expression for the average 
of an operator A in the canonical ensemble, that is, at a temperature T = l/kslS. 

{A) =Y,Pn{n\A\n) (23.8) 

n 

Inserting the expression for from eq. (23.5), this becomes 

(A) = ^ ^(n|A|n) exp(-/3A„) (23.9) 

n 

A particularly important case is the average energy. 

U = (H) = ^^{n\H\n)exp{-l3En) = ^ ^ exp(-/3A„) (23.10) 

n n 

23.3 The Quantum Mechanical Partition Function 

Although the partition function Z was introduced simply as a normalization factor 
in eq. (23.6), it turns out to be very useful in its own right in the same way that the 
classical partition function was found to be useful in Chapter 19. As in the classical 
case, the partition function in eq. (23.6) depends on the temperature as /3 = l/fc^T. 
If we take the derivative of the logarithm of the partition function, we find 
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V,N 


If we compare this equation to the thermodynamic identity 


(23.11) 


[ dim 

V 5/3 


= U 

V,N 


(23.12) 


we see that we can integrate it to obtain 

In Z = -f3F + f{V,N) (23.13) 

in which the function f{V,N) has not yet been determined. 

To determine the function f{V,N), first consider the derivative of 

F = -hT[lnZ-f{V,N)] (23.14) 


with respect to temperature. 


(i) ^ ^ 

= -kB[-PF]-hT(^-j^^ i-U) 

= ^(F-U)=^iU-TS-U) = -S 

Clearly, the function f{V,N) has no effect on either the energy or the entropy. 
Next consider the derivative of F with respect to volume. 


dV ) B jq 


-^\nZ- ■^f{V,N) 
dV dV’^ ’ ’ 


(23.16) 


= -ksT ^ ^ (-^^) exp(-/^E„) - ksT% 


dV 




^)exp(-/3i3„)-fcBT|^ 


Now we need the relationship between the pressure the system would have in an 
eigenstate, and the partial derivative of the energy of that eigenstate with respect to 
the volume. To avoid confusion with defined earlier in this section, we will denote 
the pressure of the system in an eigenstate by P{\n)). 


P{\n)) 


(dE„\ 

\ 51^ / x,N 


(23.17) 
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Combining eqs. (23.16) and (23.17), we find 

P3.18) 

where P is the average pressure. Since P must be given by the weighted average of 
the pressures in the eigenstates, 

P = i ^ P(|n)) exp(-/3P„) (23.19) 

n 

the partial derivative of f{V,N) with respect to V must vanish; / can only be a 
function of N. 


f{V,N) = f{N) (23.20) 

At this point, we would need to examine the 7V-dependence of the energy eigen¬ 
values, but this is best done later in the context of the discussion of the quantum 
ideal gas and the exchange of particles with a reservoir. For the time being we will 
tentatively assign the value zero to the function f{N). This will turn out to be a valid 
choice for fermions or bosons, but not for distinguishable particles. 

The (tentative) choice of f{N) = 0 has the great advantage of simplifying 
eq. (23.13) and giving it the same form as in classical statistical mechanics. 


lnZ = -/3P (23.21) 

or 

Z = exp(-/3F) (23.22) 


As noted in the first box in Section 23.1, the case of distinguishable particles has 
to be handled rather differently. We will see that the proper choice for a system of 
distinguishable particles is f{N) = — ln(iV!). 


23.4 The Quantum Mechanical Entropy 

The expression for the entropy can also be written in another form, which is both 
useful and revealing. 
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As in eq. (23.15), take the derivative of the free energy with respect to temperature 
to find the (negative) entropy, but now set f{V,N) = 0. 




V,N 


-ks In ^ ^ X! exp(-/3.Bn) 

n 


(23.23) 


In the second line of this equation we have used eq. (23.6) to express the derivative of 
the partition function in terms of a sum. Now recall from eq. (23.5) that 

P„ = i exp(-/3P„) (23.24) 

or 

In P„ = - In Z -/3£;„ (23.25) 


and, of course. 


E = I E = 1 (23.26) 


Eq. (23.23) can now be written as 

In Z ^ P„ - ^ ^ (-£;„) exp(-/3£’„) 

n n 

= -kB^^iPnlnZ + ^Er,Pn] 

n 

= ks'^Pn In Pn = -S, 



which gives us an alternative expression for the entropy. 

S=-kBY.PnlnPn 

n 


(23.27) 


(23.28) 


Eq. (23.28) is often taken as a fundamental starting point in books on statistical 
mechanics. Its simplicity is certainly an advantage of this choice. Nevertheless, the 
validity of this equation is not really obvious without a derivation such as the one 
given above. 
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23.5 The Origin of the Third Law of Thermodynamics 

An important consequence of eq. (23.28) is that since 1 > > 0 and lnP„ < 0, the 

entropy is always positive. This implies that the limit of the entropy as the temperature 
goes to zero is non-negative. Since the entropy at T = 0 must be a constant, and not 
minus infinity as it is in classical statistical mechanics, this establishes the Nernst 
Postulate, or Third Law of Thermodynamics, as a general consequence of quantum 
mechanics. 

To investigate the value of the entropy at T = 0, we note that 


Pn 


1 

z 


exp(-/3A„) 


exp(-/3A„) 

E„exp(-/3£;„) 


(23.29) 


If we express the partition function in terms of energy levels indexed by i, we can 
write 


Z = Q{i) exp{—PEg) 


(23.30) 


where n(£) is the degeneracy of the £th level. 

Let £ = 0 be the lowest energy level, so that £1(0) is the degeneracy of the ground 
state. Eq. (23.30) can then be written as 


Z = £1(0) exp{-PEo) 


1 ( 1 ^ 1 exp(-/3(E^ - Eo)) 


(23.31) 


Since £ = 0 is the lowest energy level, Ei — Eq > 0 for all £ > 0. That implies that as 
T ^ 0, the ground state probability, Pq, is given by 


lim Pq = lim 
T^O T^O 


1 


exp(-/3£'o) 


£1(0) exp(-/3£'o) 1 -£ J2e>o (^) exp{-P{Ei - Eq)) 


Sl(0) ft’o 

1 


£ 1 ( 0 ) 


(23.32) 


and the probability of any higher level with Ei > Eq, is 

exp{-f3Ei) 


lim Pi = lim -- 7 ——^- 

T^o T^o exp(-/3Eo) 1 -£ Er>o (Sw) exp(-/3(Ef' - Eq)) 


exp{-P{Ei - Eq)) 

£ 1 ( 0 ) 


lim 

T^O 


1 + X! f 73^ ) exp(-/3(E^' - Eq)) 


£'>0 


V£1(0) 


= 0 


(23.33) 
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Now return to eq. (23.28) for the entropy in terms of the set of ‘probabilities’ {Pn}- 
Since we have the general result that lim 2 ;^o( 2 ;lna;) = 0, we can see that 

^im S'(T) = -ke ri(0) In = kg In 0(0) (23.34) 

If the ground state is non-degenerate, 0(0) = 1, and S{T = 0) = 0. Otherwise, it 
is a positive number. 

We are often interested in the entropy per particle, rather than the total entropy. 
At zero temperature, S{T = 0)/iV, is given by 

S(T = 0) kginruo) 

= N 

If 0(0) does not depend on the size of the system, the entropy per particle vanishes 
as the system size diverges. For a finite but macroscopic system, the entropy per 
site at zero temperature is non-zero, but immeasurably small. This is the origin of 
the Planck formulation of the Nernst Postulate (or Third Law of Thermodynamics), 
which requires the entropy to be zero at T = 0. 

However, there is another possibility. Suppose that the degeneracy of the ground 
state depends on the size of the system as 

n(0) = (23.36) 


where a > 1 is some positive constant. Then the zero-temperature entropy per particle 
is given by 


S{T = 0) 
N 


kB In 
N 


fcs In a > 0 


(23.37) 


This would violate the Planck formulation of the Nernst Postulate, but Nernst’s 
original formulation would still be valid. 

The possibility suggested by eq. (23.37) actually occurs, both in model calcu¬ 
lations and real experiments. The essential feature of a system that exhibits 
S{T = 0)/N > 0 is a disordered ground state. This is true of normal window glass, 
and has been confirmed by experiment. It is also true of an interesting class 
of materials called ‘spin glasses’, which are formed when a small fraction of a 
magnetic atom (such as iron) is dissolved in a non-magnetic material (such as 
copper). For reasons that go beyond the scope of this book, the ground state of 
this system is highly disordered, leading to a large number of very low-lying energy 
levels. Although the ground-state degeneracy for these materials does not strictly 
follow eq. (23.36), the predicted zero-temperature limit S{T = 0)/A^ > 0 is found 
experimentally. There are approximate models of spin glasses for which eq. (23.37) 
can be shown to be exactly correct. 
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23.6 Derivatives of Thermal Averages 

Since the partition function depends on the temperature in a simple way, we can 
express derivatives of thermal averages with respect to T or /3 in terms of fluctuations. 
This extremely useful mathematical technique has already been discussed for the 
classical case in Section 19.8. 

For simplicity, begin with derivatives with respect to /3. Eq. (23.9) gives the formal 
expression for the thermal average {A). 

(^) = ^ ^(n|yl|n)exp(-/3F;„) (23.38) 


The partial derivative of (^) with respect to (3 gives the equation 

^(-^) = ^^{n\A\n){-ErC}eyiv{-PEn) 

= -[{AE) + {A){E)] 


(23.39) 


This result is completely general. 

Applying eq. (23.39) to the thermal average of the energy, we find 


A 


{H) 




[{E^) - (A)2] 


So we find that the derivative of the energy with respect to /3 is given by the negative 
of the variance of the energy. 

If we apply this equation to the calculation of the specific heat, we find 


1 dU _ 1 df3dU 


1 

NkeT^ 


[{E^) - {E)^] 


(23.40) 


The proportionality between the specific heat and the variance of the energy is exactly 
the same in quantum and classical mechanics. (We derived the classical version in 
Section 19.9.) 

The relationship between thermal fluctuations and thermodynamic derivatives is 
both deep and powerful. Using it can make seemingly difficult calculations become 
very easy. 


23.7 Factorization of the Partition Function 

The best trick in quantum statistical mechanics corresponds directly to the best trick 
in classical statistical mechanics, which was discussed in Section 19.12. In both cases. 




Factorization of the Partition Function 267 


the form of the Hamiltonian allows us to factor the partition function, transforming a 
many-dimensional problem into many low-dimensional problems. 

It is quite common to encounter problems in which the Hamiltonian can—at least 
approximately—be written as a sum of N terms, 

N 

H = (23.41) 

where the terms Hj all commute with one another. If this is true, the eigenvalue 
equation for Hj is 


H,\n,) = Er.^\n,) (23.42) 

and the eigenvector of the full system can be written as 

N 

\n) = n W) (23.43) 

k^l 


where we have used the index n to denote the full set of quantum numbers. 


'n- = {nj\j = 1,...,A^} 
Since the term Hj only acts on \nj), 


(23.44) 


N 

Hj\ n) = Er,, n l^fe) = \n) (23.45) 

k^l 

The full eigenvalue equation becomes 

N N N N N 

H\n) n (23.46) 

j—1 k—1 j—1 k—1 j—1 

where 


N 

En = y] Enj 
i=i 


(23.47) 
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We can use eq. (23.47) to cast the partition function in a very convenient form. 

N 

Z=Y^ exp{-p'^En^) 

{rij} j^l 

N 


= exp{-f3En^) ■■'Y exp(-/3£^n2) X! exp(-/3£;ni) 

riN ri2 Til 

N / 

= n( 

t = l \ rai 


(23.48) 


Since this equation is extremely useful, we will repeat it without the intermediate 
steps. 


N N / \ 

z=Y = n ( ( 23 . 49 ) 

This form of the partition function allows us to factor it into a product of terms, 
each of which is much easier to evaluate than the original expression. Just as for the 
corresponding equation in classical mechanics (see Section 19.12), difficult problems 
can become very easy by using eq. (23.49). 


Because of the importance of eq. (23.49) and the frequency with which it is used, 
we should call attention to a mental hazard associated with it that can catch the 
unwary. In eq. (23.49) we are exchanging a sum and a product. 

N N 

En —HE 

= l j=l rij 

However, the sums on the right and left sides of the equation are not over the same 
quantum numbers. The sum on the right, only over the quantum numbers 

associated with the term Elj in the Hamiltonian. The sum on the left, X]{r!, }> 
over the set of all quantum numbers for the entire system. In the heat of battle 
(while taking a test), it is not unusual to forget to write the indices of sums and 
products explicitly. Omitting the indices is always a bad way to save time, but it 
can be especially dangerous when using eq. (23.49). 


We are now in a position to see why eq. (23.49) is so valuable. From eq. (23.21), the 
free energy is given by the logarithm of the partition function. When the Hamiltonian 
has the form given in eq. (23.41), the free energy becomes particularly simple. 
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N 


F = —ksTlnZ = —ksTlw. En exp(-/3i;„J 



=-ksT y^exp(-/3£;„J 



(23.50) 


Since the sums in the last line of this equation are only over a single quantum number, 
rather than all combinations of N quantum numbers, they are much easier to carry 
out. 

If the spectrum of eigenvalues is the same for every term Hj, eq. (23.49) simplifies 
further. 


F = -fcBTA^ln^exp(-/3£;„J 


(23.51) 


In the following sections we will discuss the simplest examples of component 
systems that we might find after factorization. These examples are important because 
they occur repeatedly in quantum statistical mechanics. We will see applications of 
the same mathematical forms in the analysis of magnets, vibrations in crystals, black- 
body radiation, fermions, and bosons. Consequently, these examples are to be studied 
very carefully; if you know them well, most of the mathematics in the rest of the book 
will look familiar. 

23.8 Special Systems 

There are a number of many-body systems for which the partition function can be 
easily found because they have a small number of energy levels. For such systems, it 
is an easy matter to sum up a small number of terms to obtain the partition function. 

There are two classes of systems that are so important in the development of 
statistical mechanics that we will devote the remaining sections in this chapter to 
their analysis. These systems are the simple harmonic oscillators and any system with 
only two energy eigenstates. 

23.9 Two-Level Systems 

The simplest imaginable quantum system would have only one state—but it would 
not be very interesting. 

The next simplest system has two quantum eignstates. There are two forms in 
which we will encounter a two-level system. The mathematics is essentially the same 
in both cases, but since the usual ways of expressing the results differ, they are both 
worth studying. 
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23.9.1 Energies 0 and e 

The first form for a two-level system assigns the energies 0 and e to the levels. Letting 
the quantum number n take on the values of 0 and 1, the Hamiltonian is simply 

H = en (23.52) 

The partition function contains just two terms. 

1 

Z = exp(—/3en) = 1-1- exp(—/3e) (23.53) 

n—0 

The free energy and the energy are easily found. 

F = -keTlnZ = -kBTln{l + exp{-Pe)) (23.54) 


dpF eexp(—/3e) e 

5/3 1-1- exp(—/3e) exp(/3e) -I- 1 


(23.55) 


Note that the average value of the quantum number n (average number of excitations 
in the system) is given by 


n 


= exp(-/3en) = ^ 


n—0 


0 -I- exp(—/3e) 1 

-I- exp(-/3e) exp(/3e) -I-1 


which is consistent with the average value of the energy. 

1 1 ^ 

'^exp(-/3en) = Y^-~ ^ 


U = (H) = ^ iJ^ exp(-/3en) = ^ en^ exp(-/3en) = e{n) 


(23.56) 


(23.57) 


n—0 


n—0 


It will be very important to remember the expression for (n) given in eq. (23.56), 
and especially the positive sign of the exponent in the denominator. Since the usual 
Boltzmann factor exp(—/3i7) contains a negative sign, confusing the sign is a well- 
known mental hazard. 


The entropy is given, as usual, hy S = {U — F)/T. It will be left as an exercise to 
find the zero-temperature limit of the entropy, as well as its dependence on e. 

23.9.2 Spin One-Half 

The second form in which we encounter two-level systems is related to magnetism. 
If the operator cr takes on the values -1-1 and —1 (ignoring factors of Planck’s constant), 
the Hamiltonian 


H = —ha 


( 23 . 58 ) 
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corresponds to a spin-one-half magnetic moment in an magnetic field h. In this 
equation h has units of energy, which makes life easier. 

The notation in eq. (23.58) does raise the possibility of confusion between the 
magnetic field and Planck’s constant. I apologize, but beg to be excused because it 
is a very common notation. Fortunately, Planck’s constant almost never appears in 
the literature that discusses this model. We will use h to denote the magnetic field 
rather than Planck’s constant only in this section and in Chapter 30. 


The partition function again contains just two terms. 


Z = ^ exp(—/3(—/icr)) = exp{/3h) + exp(—/3ft,) 

<T = ±1 


The free energy and the energy are again easily found. 


F = —fc^Tln Z = —ftsTln [exp(/3ft) -I- exp(—/3ft)] 


d{PF) ^exp(/3ft) — exp(—/3ft) 

9/3 exp(/3ft) -I- exp(—/3ft) 


—ft tanh(/3ft) 


Note that the average value of a (average magnetization) is given by 


cr^exp(/3ftcr) 

(T — zkl 


exp(/3ft) — exp(—/3ft) 
exp(/3ft) -I- exp(—/3ft) 


tanh(/3ft) 


(23.59) 


(23.60) 

(23.61) 


(23.62) 


which is consistent with the average value of the energy. 

It is traditional to use the hyperbolic tangent in magnetic problems and the 
exponential sums for other two-level systems. The two forms are, of course, equivalent. 
The mathematical expressions 


tanh X = 


e — e 


1 - e-2^ 
1 -k e-2^ 


(23.63) 


are good choices for memorization. 


23.10 Simple Harmonic Oscillator 

The Hamiltonian of a simple harmonic oscillator (SHO) in one dimension is given by 


1 7 )^ 1 

H = -Kx^ + -— = -Kx^ — -— 

2 2m 2 2m dx^ 


(23.64) 


where m is the mass of the particle and K is the spring constant. 

The energy spectrum for a simple harmonic oscillator (SHO) has an infinite number 
of states, labeled by a quantum number that takes on non-negative integer values, 
n = 0,1,..., oo. 







272 Quantum Canonical Ensemble 


En = Hu! (23.65) 

The angular frequency, to, in eq. (23.65) is identical to the classical value. 

(23.66) 

Even though there is an infinite number of states, the uniform spacing makes the 
evaluation of the partition function only slightly more difficult than for a two-level 
system. 



Z = exp {—l3Huj{n + 1/2)) 

n—0 

exp(—/3Sa;/2) 

1 — exp{—f3Huj) 


(23.67) 


Sums of the form 


OO 


E-- 

n—O 



occur frequently in quantum statistical mechanics. The savvy student should expect 
it to appear on tests—and not only know it, but know how to derive it. 


For high temperatures (small /3), we expect the partition function to go to the 
classical value, Z^ass = l/(^/3w), as found in Section 19.13. 

^ ^ exp{-^fuj/2) 

1 — exp{—PHuj) 

1 

^ 1 n- PHuj/2+ ■■■\ 

I3hu} E ~ l3hjj/2. + ■ ■ ■) 

1 

f3hLU 

This agreement between the classical and quantum results for an SHO is the basic 
justification for the inclusion of the factors involving 1/h (where h is Planck’s constant) 
in the classical definitions of the entropy in eqs. (7.3) and (7.30), and the partition 
function in eq. (19.27). 


{1- ptiuj/2 + ---) 


1 - 




(23.68) 
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Returning to the full quantum partition function given in eq. (23.67), we can easily 
obtain the free energy and the energy. 


F 


U 


— ksT In Z 

—kBTlnexp{—j3huj/2) + kBTln{l — exp{—(3hui)) 
-fvjj + kBT\n{l — exp{—jShuj)) 

dim 

dp 


^fuv + 
^-hio + 


huj expi—phuj) 
1 — exp{—phui) 

Hu 

expiPHu) — 1 


(23.69) 


(23.70) 


The form of the energy in eq. (23.70) can be compared with the formal expression 
for the average of the energy. 


(En) 


1 

2 


Hu + Hu{n) 


(23.71) 


Either from this comparison or from a direct calculation, we can see that the average 
number of excitations of the SHO is given by the expression 


expiPhu) — 1 


(23.72) 


As it was for the average number of excitations in a two-level system in eq. (23.56), 
it will be very important to remember the expression for (n) in eq. (23.72) for a 
quantum SHO. Here again, the positive sign of the exponent in the denominator 
can be easily forgotten, which would have a negative effect on your grades. 


23.11 Einstein Model of a Crystal 

Our first application of the quantum simple harmonic oscillator is Einstein’s model of 
the vibrations of a crystal. Einstein made the approximation that all atoms except one 
were fixed at their average locations. He then approximated the potential seen by the 
chosen atom as being parabolic, so that he could treat the motion of the remaining 
atom as a three-dimensional SHO. 

Assuming for simplicity that the quadratic terms are isotropic, the Hamiltonian 
for the one moving particle is 


Hi 





2m 


(23.73) 
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where the subscript 1 indicates that this Hamiltonian only applies to the one special 
particle. 

The partition function for this system factorizes into three identical terms, each 
given by the partition function of the one-dimensional SHO from eq. (23.68). 


/ exp{—phuj/2) 

\ 1 — exp{—l3hu))) 


(23.74) 


The energy is just three times the energy of a single one-dimensional SHO. 



3huj 

exp{f3Huj) — 1 


(23.75) 


For N atoms in the crystal, the total energy is just N times the energy of a single 
atom. 


3 SNhhj 

Un = -Nhuj -\ --- 

2 exp(phuj) — 1 


(23.76) 


The specific heat at constant volume is then given by the usual derivative with 
respect to temperature. 


m _ 1 f9UN\ 
^ N \ dT ) 

1 df3 d 


V,N 


SNHuj 


N dT dp yexp^pHuj) — 1 


1 


Shuj d 

dp \exp{ptujj) — 1 

'ihjj —Hu! exp{pHuj) 
{exp{phhj) — 1)2 


= 3fcB 


tujj \ ^ exp{ptiuj) 
ksT ) {exp{phuj) — 1)2 


(23.77) 


The final expression for cv{T) in eq. (23.77) has interesting properties at high and 
low temperatures. 

At high temperatures (small /3), the factor of exp^phio) in the numerator goes to 
one, while the expression in the denominator can be expanded. 


{exp{phuj) — 1 )^ = (1 + PftuJ — 1 )^ ^ {phioY 

Using this to find the high-temperature limit of cy(T) gives: 




= 3^5 


(23.78) 


(23.79) 
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This constant value of the specific heat at high temperatures is just the well-known law 
of Dulong and Petit. It is identical to the specific heat of the corresponding classical 
model. 

At low temperatures (large f3), the factor of exp{l3tiuj) becomes extremely large, 
and the ‘—1’ in the denominator can be neglected. 

cv{T) —f Sks (/3Sw)^ exp{—f3fiuj) (23.80) 

Even though the factor of 0^ diverges at low temperature, the factor of exp(— 
goes to zero much faster. The result is that the specific heat in the Einstein model 
goes to zero rapidly as the temperature goes to zero, as it must according to the Third 
Law of Thermodynamics. 

The Einstein model was a great success in explaining the qualitative features of 
the specific heat of crystals at low temperatures and the quantitative behavior at high 
temperatures. Nevertheless, the specific heat of real crystals does not go to zero as 
rapidly as predicted by eq. (23.80). Instead, a behavior is seen in insulators, while 
the specific heat of metals is linear in T at low temperatures. The explanations of 
these observations will be given in Chapters 25 and 28. 

The next chapter discusses black-body radiation, which might seem to be a bit 
of a detour. However, it turns out that the mathematical form of the equations we 
will find is very similar to, but simpler than, the equations needed to explain the low- 
temperature behavior of the specific heat of crystals, which will follow in Chapter 25. 


23.12 Problems 


Problem 23.1 

Quantum statistical mechanics: A spin in a magnetic field 

Consider a magnetic moment with spin-one-half in a magnetic field. Being a lazy 
theorist, I prefer not to write h/2 repeatedly, so I will use units in which the spin 
(T = ±1. I will also choose the units of the magnetic field such that the energy of the 
system is just 


E = —her 

The system is in contact with a heat reservoir at temperature T. 

1. Calculate the probability of the spin having the value -1-1. 

2. Calculate the average magnetization m = {a). Express your answer in terms of 
hyperbolic functions. 

3. Calculate the two leading terms in a high-temperature expansion of the magne¬ 
tization in powers of /9 = l/ksT. 
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4. Calculate the leading (non-constant) term in a low-temperature power series 
expansion of the magnetization in some variable at low temperatures. (Hint: 
it will be in the form of an exponential.) 

5. Calculate the average energy. Plot it as a function of temperature. 

6. Calculate the specific heat. Plot it as a function of temperature 

7. Calculate the magnetic susceptibility. 

dm 

~dh 


Problem 23.2 

A computer simulation of a two-level system Quantum statistical mechan¬ 
ics: A spin in a magnetic field 

Consider the same magnetic moment with spin-one-half in a magnetic field that we 
looked at in an earlier problem. The units of the magnetic field are again chosen such 
that the energy of the system is just 


E = —ha 

The system is in contact with a heat reservoir at temperature T. 

For this problem, instead of doing an analytic calculation, we will carry out a 
computer simulation using the Monte Carlo method. Although this might seem to 
be the hard way to do it, we will see later that the method can be generalized to solve 
problems that do not have analytic solutions. 

1. Write a computer program to simulate this two-level system. The program should 
calculate the thermal probability of being in each state, as well as the average 
magnetization and the magnetic susceptibility. Have the program print out of the 
theoretical values for comparison. 

2. Use your program to calculate the magnetization and magnetic susceptibility for 
a set of ‘interesting’ values of the magnetic field and temperature. 


Problem 23.3 

Quantum statistical mechanics: Another two-level system 

Although a two-level system might seem very simple, it is very important and occurs 
frequently in various guises. Here is another form that we will see often. 

A system only has two states, which are both non-degenerate. The energies of these 
two states are E = 0 and E = e > 0. 

The system is in contact with a heat reservoir at temperature T. 

1. Calculate the probability of being in the excited state. 

2. Calculate the average energy. Sketch it as a function of temperature. 

3. Calculate the average specific heat. Sketch it as a function of temperature. 

4. Calculate the two leading terms in a high-temperature expansion of the energy 
in powers of /3 = l/ksT. 
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5. Calculate the leading (non-constant) term in a low-temperature power series 
expansion of the energy in some variable at low temperatures. (Hint: the variable 
will be in the form of an exponential.) 

Problem 23.4 

Quantum statistical mechanics: Return of the simple harmonic oscillator 

The energy levels of a quantum SHO are given by 



where n = 0,1, 2,..., 



and 



1. Calculate the QM partition function. 

2. Calculate the probability that a measurement of the energy will find the QM 
SHO in the n-th eigenstate. 

3. Calculate the average energy. 

4. Calculate the specific heat. 

5. Calculate the high-temperature limit of the specific heat. 

6. Calculate the leading term in a low-temperature expansion of the specific heat. 
(You should be able to figure out what a good quantity to expand in would be.) 

Problem 23.5 

A Monte Carlo computer simulation of a quantum SHO 

As we’ve seen before, the energy levels of a quantum SHO are given by 



where n = 0,1, 2,..., 



and 
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This time we shall carry out a Monte Carlo simulation of this system. There are, of 
course, an infinite number of possible states. However, the probability of high-energy 
states is very small, making the simulation feasible. 

1. Write a program to do a Monte Carlo simulation of a single quantum SHO. Label 
the state of the system with the quantum number n, as in the equations above. 
Let one MC step consist of an attempted move to increase or decrease n by ±1. 
Remember to reject any attempted move that would make n negative. 

2. Compute the energy and specific heat (the latter from fluctuations) for a range 
of interesting temperature. For convenience, you may take hco = 1.0. Have the 
program print out the exact values for comparison. 

Problem 23.6 

Quantum statistical mechanics: A many-spin system 

Consider a macroscopic crystal with a spin-one quantum mechanical magnetic moment 
located on each of N atoms. Assume that we can represent the energy eigenvalues of 
the system with a Hamiltonian of the form 


N 



n—1 


where each (t„ takes on the values —1, 0, or -|-1, and U is a constant representing a 
‘crystal field’. The entire system is in contact with a thermal reservoir at tempera¬ 
ture T. 

1. Calculate the partition function for this system. 

2. Calculate the free energy of this system. 

3. Calculate the quantity 



4. Calculate the entropy per spin of this system. 

5. Determine whether this system satisfies the Nernst Postulate for all values of the 
parameters. 

Problem 23.7 

Quantum statistical mechanics: A many-spin system (This is a more diffi¬ 
cult version of the previous problem) 

Consider a macroscopic crystal with a spin-one quantum mechanical magnetic moment 
located on each of N atoms. Assume that we can represent the energy eigenvalues of 
the system with a Hamiltonian of the form 
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N 


N 




n—l 


n—1 


where each cr„ takes on the values —1, 0, or +1, and B and D are constants representing 
an external magnetic field and a “crystal field,” respectively. The entire system is in 
contact with a thermal reservoir at temperature T. 

1. Calculate the partition function for this system. 

2. Calculate the free energy of this system. 

3. Calculate the magnetization per spin 



4. Calculate the entropy per spin of this system. 

5. Determine whether this system satisfies the Nernst Postulate for all values of the 
parameters. 

Problem 23.8 

A diatomic ideal gas Quantum statistical mechanics—but only when 
needed 

Consider a dilute diatomic gas that you can treat as ideal in the sense of neglecting 
interactions between the molecules. We shall assume that the molecules consist of two 
point masses with a fixed distance between them. They can rotate freely, but they 
cannot vibrate. 

The center of mass motion (translational degrees of freedom) can be treated 
classically. However, the rotational degrees of freedom must be treated quantum 
mechanically. 

The quantum mechanical energy levels take the form e(j) = j(j + l)eo, where 
j = 0,1,2,... and the degeneracy of the j-th level is given by g{j) = 2j + 1. (You can 
peek at a QM text to find the value of Cq, but you do not need it for this assignment.) 
Although you do not need it for this assignment, the parameter Cq is given by 



where I is the moment of inertia of a molecule. 

The whole system is in equilibrium with a thermal reservoir at temperature T. 

1. Write down the canonical partition function, treating the translational degrees of 
freedom classically and the rotational degrees of freedom quantum mechanically. 

2. Evaluate the energy and the specific heat for both high and low temperatures. 

3. Sketch the energy and the specific heat as functions of temperature, indicating 
both the high- and low-temperature behavior. 
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Problem 23.9 

Another diatomic ideal gas—entirely classical this time, but in two dimen¬ 
sions 

Consider a classical, dilute, diatomic gas in two dimensions. The gas is again ideal 
in the sense of neglecting interactions between the molecules. Each molecule consist 
of two point masses. Although there are no interactions between different molecules, 
there is an interaction between the atoms in the same molecule of the form 


V{r) 


Jln(r) r > a 
oo r < a 


The gas is contained in a two-dimensional ‘box’ of area A = L^. The whole system is 
in equilibrium with a thermal reservoir at temperature T. 

1. What is the classical Hamiltonian for this system? 

2. Write the canonical partition function as an integral over phase space. 

3. Calculate the partition function in closed form, under the assumption that the 
molecule is much smaller than the box it is in. That is, let the limits on the 
integral over the separation between the atoms in a molecule extend to oo. 

4. This model is only valid for low temperatures. At what temperature do you expect 
it to break down? 

5. Now calculate the average square separation (r^) between the atoms in a molecule. 


Problem 23.10 

Two-level quantum systems 


1. Consider a simple set of N two-level subsystems. The subsystems are all inde¬ 
pendent, and each has two allowed states with energies 0 (ground state) and e 
(excited state) so that the full Hamiltonian can be written as 


N 

H = Y. 

n—1 


where E{n) = 0 or E(n) = e The entire system is in thermal equilibrium at 
temperature T. 


1. At what temperature is the average total energy equal to 


2. At what temperature is the average total energy equal to 


2. Suppose the subsystems in the previous problem had the same ground state with 
energy 0, but different values of the energy in the excited states. Assume that 
the energy of the excited state of the n-th subsystem is ne. 
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1. What is the average energy of the total system? 

2. Suppose we are in the low-temperature regime, for which kgT <C E^ax- Calcu¬ 
late the average energy. You may leave your answer in terms of a dimensionless 
integral, but you should obtain the temperature dependence. 

3. What is the heat capacity of the total system? 


24 

Black-Body Radiation 


The spectral density of black body radiation ... represents something absolute, 
and since the search for the absolutes has always appeared to me to be the 
highest form of research, I applied myself vigorously to its solution. 

Max Planck, German physicist (1858-1947), Nobel Prize 1918 


24.1 Black Bodies 

In physics, the expression ‘black body’ refers to an object that absorbs all radiation 
incident on it and reflects nothing. It is, of course, an idealization, but one that can 
be approximated very well in the laboratory. 

A black body is not really black. Although it does not reflect light, it can and does 
radiate light arising from its thermal energy. This is, of course, necessary if the black 
body is ever to be in thermal equilibrium with another object. 

The purpose of the current chapter is to calculate the spectrum of radiation 
emanating from a black body. The calculation was originally carried out by Max 
Planck in 1900 and published the following year. This was before quantum mechanics 
had been invented—or perhaps it could be regarded the first step in its invention. In 
any case, Planck investigated the consequences of the assumption that light could only 
appear in discrete amounts given by the quantity 


= hv = two (24.1) 

where v is the frequency, u = 2TTiy is the angular frequency, h is Planck’s constant, and 
h= h/2TT. This assumption is well accepted today, but it was pretty daring in 1900 
when Max Planck introduced it. 

24.2 Universal Frequency Spectrum 

If two black bodies at the same temperature are in equilibrium with each other, the 
frequency spectrum must be the same for each object. To see why, suppose that two 
black bodies, A and B, are in equilibrium with each other, but that A emits more 
power than B in a particular frequency range. Place a baffle between the objects that 
transmits radiation well in that frequency range, but is opaque to other frequencies. 
This would have the effect of heating B and cooling A, in defiance of the Second Law 
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of Thermodynamics. Since that cannot happen, the frequency spectrum must be the 
same for all black bodies. 

Since the radiation spectrum does not depend on the object, we might as well take 
advantage of the fact and carry out our calculations for the simplest object we can 
think of. 

24.3 A Simple Model 

We will consider a cubic cavity with dimensions L x L x L. The sides are made of 
metal and it contains electromagnetic radiation, but no matter. Radiation can only 
come in and out of the cavity through a very small hole in one side. Since the radiation 
must be reflected off the walls many times before returning to the hole, we can assume 
that it has been absorbed along the way, making this object—or at least the hole—a 
black body. 

The only thing inside the cavity is electromagnetic radiation at temperature T. 
We wish to And the frequency spectrum of the energy stored in that radiation, which 
will also give us the frequency spectrum of light emitted from the hole. 

24.4 Two Types of Quantization 

In analyzing the simple model described in the previous section, we must be aware of 
the two kinds of quantization that enter the problem. 

As a result of the boundary conditions due to the metal walls of the container, 
the frequencies of allowed standing waves are quantized. This is an entirely classical 
effect, similar to the quantization of frequency in the vibrations of a guitar string. 

The second form of quantization is due to quantum mechanics, which specifies 
that the energy stored in an electromagnetic wave with angular frequency w comes in 
multiples of huj (usually called photons). 

The theory of electrodynamics gives us the wave equation for the electric held E{f) 
in a vacuum. 


In eq. (24.2), 


TJ^E{r,t) 


I d'^E{f,t) 


_5 \ 


and 


q2 q2 q2 

dx'^ dy"^ dz'^ 


(24.2) 


(24.3) 


(24.4) 


The solutions to eq. (24.2) must satisfy the boundary conditions that the compo¬ 
nent of the electric held parallel to a wall must vanish at that wall. 
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Using the symmetry of the model, we find that there are solutions of the following 
form. 


Ex{r,t) = Ex;cos {kxX) sin (kyu) sin (kzz) (24.5) 

Ey{f, t) = Ey^o sin(a;t) sin (k^x) cos {kyy) sin (k^z) (24.6) 

Ez{r,t) = E 2 ^oSin{ujt) sin {kxx) sin {kyy) cos (k^z) (24.7) 

The values of ifx.o, Ey^o, and Ez^o are the amplitudes of the corresponding components 
of the electric field in the cavity. 

Eq. (24.5) through (24.7) were written to impose the boundary condition that the 
parallel components of the electric field vanish at the walls of the cube where x, y, 
or z is equal to zero. To impose the same boundary condition at the remaining walls, 
where x, y, or z is equal to L, we have the conditions 


kxL — TLx^ 

(24.8) 

kyL TlyTT 

(24.9) 

kzL = UzTT 

(24.10) 


where rix, riy, Uz are integers. Only positive integers are counted, because negative 
integers give exactly the same solutions. 

Substituting eqs. (24.5), (24.6), and (24.7) into eq. (24.2), we find a relationship 
between the frequency and the wave numbers. 


klkykl — 


(24.11) 


This equation can also be written in terms of the integers n^,, Uy, Uz- 



I'+l 

(nyTr\ 


fnz7r\ 

2 

V L ) 


1 L J 


1 L J 

* “ C2 


(24.12) 


Clearly the value of uj must depend on the vector ft = {nx,ny,nz)., We will indicate 
the n-dependence by a subscript and solve eq. (24.12) for Wfj. 


OJf, 



(24.13) 


Taking the square root to find Wfj, we obtain 





UTTC 


(24.14) 


where n = |n|. 

Because the wavelengths that contribute significantly to black-body radiation are 
very small in comparison with the size of the cavity, energy differences between 
neighboring points in n-space are very small. This makes the frequency spectrum 
quasi-continuous and allows us to change sums over the discrete wavelengths into 
integrals. Furthermore, the dependence of the frequency on n shown in eq. (24.14) is 
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rotationally symmetric, which simplifies the integrals further, as shown in the next 
section. 


24.5 Black-Body Energy Spectrum 

The first step in calculating the black-body energy spectrum is to find the density 
of states. From the solutions to the wave equation in Section 24.4, we expressed the 
individual modes in terms of the vectors n. Note that the density of points in n-space 
is one, since the components of n are all integers. 

Pn{ri) = 1 (24.15) 


To find the density of states as a function of frequency, integrate eq. (24.15) 

over n-space. 


Puioj) 


1 

= 2- 4nn^dnS{u; 
° Jo 


ncir/L) = TT 



(24.16) 


The factor of 2 is for the two polarizations of electromagnetic radiation, and the factor 
of 1 /8 corrects for counting both positive and negative values of the components of n. 

Since each photon with frequency uj has energy Hu, the average energy can be 
found by summing over all numbers of photons weighted by the Boltzmann factor 
exp(—PHu). Since this sum is formally identical to that for the simple harmonic 
oscillator, we can just write down the answer. 




Hu 

exp{pHu) — 1 


(24.17) 


Note that eq. (24.17) does not include the ground-state energy Hu/2 that might be 
expected for a simple harmonic oscillator. The reason is a bit embarrassing. Since 
there is an infinite number of modes, the sum of the ground-state energies is infinite. 
The simplest way to deal with the problem is to ignore it on the grounds that a 
constant ground-state energy cannot affect the results for the radiation spectrum. 
That is what other textbooks do, and that is what I will do for the rest of the book. 
I suggest you do the same. 


The energy density spectrum for black-body radiation as a function of the angular 
frequency u is found by multiplying the density of states in eq. (24.16) by the average 
energy per state in eq. (24.17) and dividing by the volume V = L^. 


Uu: 


/V\ 2 Hu 

^ exp{l3Hu) — 1 


{exp{f3Hu) - 1) ^ 


(24.18) 


Knowing the energy per unit volume contained in the black-body cavity from 
eq. (24.30), and the fact that light travels with the speed of light (if you will pardon 
the tautology), the energy per unit area radiated from the hole in the cavity, Jjj, is 
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Fig. 24.1 Plot of black-body radiation spectrum, x^/{exp{x) — 1), in dimensionless units. For 
comparison with the black-body spectrum in eq. (24-19), note that x = (5huj. The integral under 
this dimensionless function was given as 7r'*/15 in eq. (24-29). 


clearly proportional to cU(V. The actual equation includes a geometric factor of 1/4, 
the calculation of which will be left to the reader. 

Multiplying by the factor of c/4 to derive the radiated power gives us the Planck 
law for black-body radiation. 


4^“"^ (47r2c2) exp(/3fej) - 1 

Fig. 24.1 shows a plot of eq. (24.19) in dimensionless units; that is, a;^/(exp(x) — 1), 
where x = (ihiLo. The function has a maximum at Xmax ~ 2.82144. 

It is very important to understand how the spectrum of black-body radiation scales 
as a function of T, which will be discussed in the following subsections. 


24.5.1 Frequency of Maximum Intensity 

Since w = x{kBT/K), the location of the maximum is 

^max ~ ^max (H ~ 2.82144 

which is proportional to the temperature, T. 

The value of j^i at its maximum is then 

^ \ {xmax ksT/hf ^ \ {keTf 

) exp{xmax)-^ J exp{Xmax) - i 




(24.20) 


(24.21) 


which is proportional to T^. Clearly, the integral over the black-body spectrum 
is proportional to x T = T'^, as expected from the Stefan-Boltzmann Law in 
eq. (24.31). 
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24.5.2 Low-Frequency Spectrum 

The low-frequency energy spectrum is found for x = (dhiu « Xmax ~ 2.82. For small 
values of w, we find 


juj = (24.22) 

/ \ 

exp{phuj) — 1 

_ [ fi \ 

\47r^c^ ) 1 — Phui — 1 
_ [ fi \ UJ^ 

phu! 

This expression is reasonable because the small uj region corresponds to phut 1, or 
kgT ^ hu}, which is the condition that the classical theory is valid. In this limit, 

(e„) ^ keT (24.23) 

independent of the value of co. The factor of cu'^ in eq. (24.22) comes from the factor 
of in eq. (24.16), which, in turn, came from the nP dependence of the surface of a 
sphere in n-space. 


24.5.3 High-Frequency Spectrum 

At high frequencies, phuj ^ Xmax > 1, so that we can make the approximation that 
exp{ptiuj) ^ 1 in the expression for the spectrum of black-body radiation in eq. (24.22). 


= 


/ \ 


( ^ ] 

\47r^c^ ) 

' exp{PHu)) — 1 ' 

i^47r^c^ J 


uj^ exp(—/3Sw) 


(24.24) 


To understand the high-frequency behavior of the energy spectrum, first note that 
two factors of to come from the rP dependence of the surface of a sphere in n-space, 
as noted in the previous subsection. The third factor of to comes from hoj, which is 
the energy of a single photon, while the factor exp{—pfiiS) gives the relatively low 
probability of a high-frequency photon being excited. 

Note that although the factor diverges as uj increases, the factor of 
exp{—phu;) goes to zero much more rapidly, giving the shape of the curve shown in 
Fig. 24.1. 
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24.6 Total Energy 

The total quantum mechanical energy in the cavity radiation at temperature T is 
given by summing up the average energy in each mode. 

U = 2 ^(es) = 2 ^ feus {expiPfuvn) - 1)”^ (24.25) 

n n 

The sum in eq. (24.25) is restricted to the positive octant in n-space in order to avoid 
double counting, and factor of two accounts for the two polarizations of light associated 
with every spatial mode. (Note that we have again omitted the ground-state energy 
for the electromagnetic modes for the same dubious reason given in Section 24.5.) 

We again use the fact that the frequency spectrum is a quasi-continuum to write 
the sum in eq. (24.25) as an integral, which we can evaluate explicitly using the density 
of states, Pu{to), found in eq. (24.16). 

pOO 

u= {e^)P^{uj)du; (24.26) 

L 

_ ( lV hoj 2 , 

^\cTtJ Jo exp(/3feu) -^ 

At this point we can simplify the equation for U by introducing a dimensionless 
integration variable. 


X = phuj 


(24.27) 


The expression for U becomes 


U = 



(24.28) 


By a stroke of good fortune, the dimensionless integral in this equation is known 
exactly. 



15 


(24.29) 


Noting that the volume of the cavity is 1^ = , the average energy per volume in the 

cavity can be given exactly. 




(24.30) 
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Since the energy of a black body is proportional to the specific heat per unit 
volume must be proportional to T^. This might not seem terribly significant now, 
but keep it in mind for later in Chapter 25, when we discuss the specific heat of an 
insulator, which is also proportional to at low temperatures. 


24.7 Total Black-Body Radiation 

Multiplying the total energy by a factor of c/4, as explained in Section 24.5, we find 
the total black-body radiation density. 


Ju 


_cU _c f 


4V 4 \ 15;i3c3 


(ksT)^ = (TbT^ 


(24.31) 


where 




(24.32) 


The constant cjb is known as the Stefan-Boltzmann constant. It is named after the 
Austrian physicist Joseph Stefan (1835-1893), who first suggested that the energy 
radiated by a hot object was proportional to T^, and his student Boltzmann, who 
found a theoretical argument for the fourth power of the temperature. The value of 
the Stefan-Boltzman constant had been known experimentally long before Planck 
calculated it theoretically in 1900. Planck got the value right! 


24.8 Significance of Black-Body Radiation 

Perhaps the most famous occurrence of black-body radiation is in the background 
radiation of the universe, which was discovered in 1964 by Arno Penzias (German 
physicist who became an American citizen, 1933-, Nobel Prize 1978) and Robert 
Wilson (American astronomer, 1936-, Nobel Prize 1978). 

Shortly after the Big Bang, the universe contained electromagnetic radiation at 
a very high temperature. With the expansion of the universe, the gas of photons 
cooled—much as a gas of particles would cool as the size of the container increased. 
Current measurements show that the background radiation of the universe is described 
extremely well by eq. (24.19) at a temperature of 2.725 AT. 


24.9 Problems 


Problem 24.1 

Generalized energy spectra 

For black-body radiation the frequency of the low-lying modes was proportional to 
the magnitude of the wave vector k. Now consider a system in d dimensions for 
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which the relationship between the frequencies of the modes and the wave vector is 
given by 


u! = Ak'^ where A and s are constants 

What is the temperature dependence of the specific heat at low temperatures? 

Problem 24.2 

Radiation from the sun 

1. The sun’s radiation can be approximated by a black body. The surface tem¬ 
perature of the sun is about 5800 ilT and its radius is 0.7 x 10® m. The distance 
from the earth to the sun is 1.5 x 10^^ m. The radius of the earth is 6.4 x 10® m. 
Estimate the average temperature of the earth from this information. Be careful 
to state your assumptions and approximations explicitly. 

2. From the nature of your assumptions in the previous question (rather than 
your knowledge of the actual temperature), would you expect a more accurate 
calculation with this data to give a higher or lower answer? Explain your reasons 
clearly to obtain one bonus point per reason. 


Problem 24.3 
Black-body radiation 

On the same graph, sketch the energy density of black body radiation it(w) vs. the 
frequency lo for the two temperatures T and 2T. (Do not change the axes so that the 
two curves become identical.) 

The graph should be large —filling the page, so that details can be seen. Small 
graphs are not acceptable. 


25 

The Harmonic Solid 


It is not knowledge, hut the act of learning, not possession but the act of getting 
there, which grants the greatest enjoyment. 

Johann Carl Friedrich Gauss, German mathematician (1777-1855) 


In this chapter we return to calculating the contributions to the specific heat of a 
crystal from the vibrations of the atoms. We have previously discussed a cruder 
approximation, the Einstein model, in Section 23.11 of Chapter 23. The harmonic 
solid is a model of lattice vibrations that goes beyond the Einstein model in that it 
allows all the atoms in the crystal to move simultaneously. 

To simplify the mathematics we will consider only a one-dimensional model in this 
book. The general extension to three dimensions does bring in some new phenomena, 
but it complicates the notation unnecessarily at this point. We will go into three 
dimensions only for particularly simple cases, in which the extension of the theory 
does not present any difficulties. 

After going through the discussion in this chapter, it should be easy to follow the 
mathematics of the general three-dimensional case in any good textbook on solid-state 
physics. 


25.1 Model of an Harmonic Solid 

A one-dimensional model of a crystal lattice is described by uniformly spaced points 
along a line. 


Rj = ja (25.1) 

The spacing a is called the lattice constant and the index j is an integer. Atoms are 
located at points 


rj = Rj + Xj = ja + Xj (25.2) 

where Xj is the deviation of the position of an atom relative to its associated lattice 
point. 
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To write an expression for the kinetic energy we will need the time derivative of 


Xj which we will indicate by a dot over the variable. 




(25.3) 


The energy of a microscopic state is given by the following expression. 

^ + (25.4) 

j 3 


The potential energy is a minimum for nearest-neighbor separations equal to the lattice 
constant, a, and is quadratic in the deviations from this optimum separation. As we 
will see, it is the assumption of a quadratic potential energy that makes this model 
tractable. 

The general strategy to solve for the properties of the harmonic crystal is to 
transform the problem from N interacting particles to N independent simple harmonic 
oscillators, which enables us to factorize the partition function as discussed in Section 
23.7. Once we have made this transformation, we need only copy the results for 
quantum SHOs from Section 23.10, and we have solved for the properties of the 
harmonic solid. 


25.2 Normal Modes 

The key to reducing the problem to independent oscillators is to Fourier transform^ 
the position variables to find the normal modes. The Fourier transform of the positions 
of the particles is given by 


Xk = N ^ Xj exp(—fA:i?j) (25.5) 

3 

where k is the (one-dimensional) wave vector or wave number. We can anticipate that 
the wave number is related to the wavelength A by 

Ott 

k=^ (25.6) 

A 

An obvious property of the Fourier transformed variables Xk is that 


x-k = xl 

where the superscript * indicates the complex conjugate. 


(25.7) 


^Jean Baptiste Joseph Fourier. Fi'ench mathematician and physicist (1768-1830). 
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25.2.1 Inverse Fourier Transformation 

The inverse of the Fourier transformation in eq. (25.5) is 

Xj = E Xfc exp(zfci?j) (25.8) 

k 

which we can confirm by direct substitution. 

Xk = ^ Xj exp{—ikRj) (25.9) 

3 

— ^ ^ Xk' exp{ik'Rj) exp{—ikRj) 

j k' 

= ^ Xk'N~^ ^ exp(i(fc' — k)Rj) 

k' j 

— ^ ( Xk' ^k,k' — Xk 

k' 

The identity, 


exp(z(fc' — k)Rj) = N6k,k' (25.10) 

3 

which was used in deriving eq. (25.9), will often prove useful in statistical mechanics. 
The validity of eq. (25.10) can be easily shown. 

exp {i{k' — k)Rj) = exp (i{n' — n) (25.11) 

3 i ^ ^ 

= y]exp (i2Tr{n' - n)j^j 

If n' = n, so that k' = k, then the sum is clearly equal to N. li n' — n = 1, the sum 
is simply adding up the N complex roots of —1. Since they are uniformly distributed 
around the unit circle in the complex plane, they sum to zero. If n' — n is any other 
integer (excepting a multiple of iV), the angle between each root is multiplied by the 
same amount, they are still distributed uniformly, and they still sum to zero. 

To complete the Fourier transform, we must specify the boundary conditions. There 
are two kinds of boundary condition in general use. Since they each have their own 
advantages, we will discuss them both in detail. 

25.2.2 Pinned Boundary Conditions 

Pinned boundary conditions for the harmonic solid are similar to the boundary 
conditions we used for black-body radiation. In that problem, the transverse electric 
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field vanished at the boundaries of the metal box. For pinned boundary conditions, 
the vibrations vanish at the boundaries of the system. 

To implement pinned boundary conditions, first extend the number of atoms in the 
model from to iV + 2 by adding atoms with indices 0 and + 1. These two atoms 
enforce the boundary conditions by being fixed (‘pinned’) at the positions iio = 0 £^nd 
Rn+1 = (N + l)a. 

The two solutions exp{ikRj) and exp{—ikRj) can be combined to give Xk oc 
sin{kRj) to satisfy the boundary condition that xq = 0 at i?o = 0- From the other 
boundary condition that xn+i = 0 at Rn+i = {N + l)a = L, we need 

kL = k{N + l)a = nn (25.12) 


or 


k = k(n) 


niT 

~L 


(25.13) 


for some integer n. 

Since we only have N atoms in the system, we only expect to find N independent 
solutions. If we define 

Ott 

K=— (25.14) 

we can see that k and k + K correspond to the same solution. 

sin((fc + K)Rj) = sin(fci?j + jaK) (25.15) 

, / „ 2-k 

= sm fciti + ja — 

\ ' a 

= sin {kRj + 2Trj) 

= sin(kRj) 

Adding any integer multiple of K to k also produces the same solution. 

For calculations, the standard choice for pinned boundary conditions is to use only 
those solutions corresponding to values of n from 1 to N. 


25.2.3 Periodic Boundary Conditions 

The other possible boundary conditions are periodic. In one dimension, this might be 
thought of as bending the line of atoms into a circle and tying the ends together, so 
that 


Rj+n = Rj (25.16) 

Note that the length of a system with periodic boundary conditions is 

L = Na (25.17) 


in contrast to the length of (N + l)a for pinned boundary conditions. 
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In two dimensions, a rectangular lattice with periodic boundary conditions can be 
viewed as a torus. It is rather diffcult to visualize periodic boundary conditions in 
three dimensions (it is called a Klein bottle), but the mathematics is no more difficult. 

The great advantage of periodic boundary conditions is that the system becomes 
invariant under translations of any multiple of the lattice constant a. The system has 
no ends in one dimension (and no borders or surfaces in two or three dimensions). 

For periodic boundary conditions, we will take solutions of the form exp(ikRj). 
The condition that the system is periodic means that 

exp {ik{Rj + L)) = exp{ikRj) (25.18) 

which implies that 

kL = 2mT (25.19) 

or 

k = k{n) = (25.20) 

for some integer value of n. Note that the values of k given in eq. (25.20) for periodic 
boundary conditions are spaced twice as far apart as the values of k for pinned 
boundary conditions given in eq. (25.13). 

As was the case for pinned boundary conditions, the solution corresponding to k 
is identical to the solution corresponding to k + K, where K is given by eq. (25.14), 
we can confirm that k and k + K correspond to the same solution. 

exp {i{k + K)Rj) = exp (^kRj + i — ja^ (25.21) 

= exp {ikRj + f27rj) 

= exp {ikRj) 


In eq. (25.21), do not forget that j is an integer index, while i = 


Note that n = 0 is a solution for periodic boundary conditions, while it is not for 
pinned boundary conditions because sin(O) = 0. The solution for n = 0 corresponds 
to moving all atoms together in the same direction. 

It is customary to take both positive and negative values of k{n), so that the central 
values of n we will use are: 

n = 0,±1,±2, ...,±(7V-l)/2 for TV odd (25.22) 


and 


n = 0,±l,±2,...,±(iV/2- l),TV/2 for N even 


(25.23) 
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For even values of N, the state corresponding to —N/2 is identical to the state 
corresponding to N/2. 

For periodic boundary conditions, the values of k lie at symmetric points in fc-space, 
and the only independent states lie between k = — iF/2 and k = if/2. This region is 
called the Brillouin Zone in honor of Leon Nicolas Brillouin (French physicist, 1889- 
1969). All information in the Brillouin Zone is repeated throughout fc-space with a 
periodicity of if = 2'k ja. 


When you take a course in solid state physics, you will find that three-dimensional 
Brillouin Zones can be rather complicated—and much more interesting. They are 
essential in understanding important properties of real materials, but they go 
beyond the scope of this book. 


25.3 Transformation of the Energy 

If we apply the Fourier transform in eq. (25.8) to the energy of the harmonic crystal 
given in eq. (25.4), we find that each Fourier mode is independent. This is the great 
simplification that justifies the bother of introducing Fourier transforms, because it 
enables us to factorize the partition function. 

We will treat the kinetic and potential energy terms separately, beginning with the 
kinetic terms. The kinetic terms are, of course, already diagonal before the Fourier 
transform, but we have to demonstrate that they are also diagonal after the Fourier 
transform. 


25.3.1 Kinetic Energy 

Using the time derivative of eq. (25.8). 

Xj = ^ ifc exp(ffci?j) (25.24) 

k 

we can carry out a Fourier transform of the kinetic energy term in eq. (25.4). 

K.E. = 

3 3 

= exp(ffcj?j)^ exp{ik'Rj'^ 

- N -771 EE Xkik' ^ exp(7(fc -I- k')Rj) 

k k' j 
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= N-^ -m EE 


(25.25) 


k k' 

= ^m'^XkX-k = ^”^E 

k k 

For the last equality, we have used eq. (25.7). 

25.3.2 Potential Energy 

For the potential energy term in eq. (25.4), separate the factors of {xj — Xj+i) in the 
sum and use eq. (25.24) to introduce Xk- 


P.E. = '^{xj - Xj+if = '^{xj - Xj+i){xj - Xj+i) 

3 3 

= ^ Xk (exp(ifci?j) - exp(ifci?j+i))^ 

X E Xk' {e-xp(ik'Rj) — exp(i/c'i?j+i))^ 

= ^ ^ Xk exp(zA:ja) (1 — exp(zfca))^ 

X Xk' exp{ik'ja) (1 — exp(ifc'a))^ 

Next, collect terms and simplify using the identity eq. (25.10). 

P.E. = l:KN-^ EE XkXk' (1 — exp(tfca)) (1 — exp(ifc'a)) 

fc k' 

X ^ exp(i(fc — k')ja) 

3 

= -K XkX-k (1 ~ exp(ifca)) (1 — exp{—ika)) 

k 


The function 

K{k) = K — exp(ifca)) (1 — exp(—ifca)) 
gives the effective spring constant for the mode with wave number k. 


(25.26) 


(25.27) 


(25.28) 
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Adding eqs. (25.25) and (25.27) together, we find the energy of the harmonic crystal 
from eq. (25.4) in terms of the Fourier transformed variables. 

E= + = (\rn\xk\'^ + ^K{k)\xk\A (25.29) 

k k k ^ ' 

The problem has been transformed from one with N variables (the positions of 
the atoms) to N problems, each with a single variable (the amplitude of the mode Xk 
with wave number k). Each mode represents a simple harmonic oscillator with mass 
m (the same as the original atomic mass) and spring constant K(k). For each mode, 
the square of the frequency is given by the usual ratio of the spring constant to the 
mass. 




K{k) 

m 


(25.30) 


25.4 The Frequency Spectrum 

To analyze the spectrum of frequencies, uj(k), we can rewrite eq. (25.28) for K(k) in 
a more convenient form. 


K{k) = K {1 — exp(ifca)) (1 — exp{—ika)) (25.31) 

= K (1 — exp{ika) — exp{—ika) + 1) 

= 2K (1 — cos(fca)) 

= 4K sin^(fca/2) 


The angular frequency is then given by 


io^ik) 


4K sm^{ka/2) 
m 


or 


uj{k) = 2d) 


sin 



(25.32) 


(25.33) 


where uj = sjKim. This spectrum is plotted in Fig. 25.1. 

For small wave number (fc <C "x ja).^ the frequency of a mode in eq. (25.33) becomes 
linear in the wave number k. 


uj(k) = 2 Cj\ sin(fca/2)| « Cbka (25.34) 

The speed of a sound wave v{k) is given by the product of the frequency v{k) times 
the wavelength for long wavelengths (small k). From eq. (25.34), we see that the speed 
of a sound waves is a constant. 
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Fig. 25.1 Frequency spectrum of a one-dimensional harmonic solid, as given in eg. (25.33). 



(25.35) 


The linearity of the function uj{k) can also be seen near the origin in Fig. 25.1. 

The generalization of these calculations to a three-dimensional harmonic crystal 
is not particularly difficult, although the mathematics becomes rather more messy 
because of the necessary introduction of vector notation. The three-dimensional form 
of the Brillouin Zone is also somewhat more complicated. However, the generalization 
of the frequency spectrum to three dimensions has a particularly simple form. 



(25.36) 


s 


The sum in this equation goes over the vectors 5 connecting a site to its nearest 
neighbors. Eq. (25.36) is valid for all lattices. 

25.5 The Energy in the Classical Model 

Since the SHOs in eq. (25.29) are not particles that can move from one system to 
another, the classical partition function given below does not contain a factor of 1/A^!. 



(25.37) 


The limits for the integrals in eq. (25.37) have been omitted (which is not usually a 
good idea) because they all go from — oo to oo. 
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Because of the diagonal form of the Fourier transformed energy, the partition 
function factors. Since all the integrals are gaussian, we can evaluate them immediately. 


Zciass = 11^ [(27rmfcBT)^/" {2ti keT/ 


h . 


(25.38) 


Simplifying this expression, we find 


where 


Zcl 


ass 


n 




Y[{pnw{k)) ^ 


uj{k) = 



(25.39) 


(25.40) 


is the frequency of the fc-th mode. 

As usual, the average energy is found by taking a derivative with respect to f3. 


^ d{l3F) 
dj3 

_ ^(^11-^c/ass) 

“ dp 

= -3iVln/3-Vln(Sw(fc)) 

k 

= = SNksT 

H 


(25.41) 


As expected, the specific heat has the constant value of 3fcB, which is the Law of 
Dulong and Petit. 

For a classical harmonic crystal, the spectrum of frequencies as a function of the 
wave number k is unimportant for equilibrium properties. In the next section we will 
see that this is not the case for a quantum mechanical harmonic crystal. 


25.6 The Quantum Harmonic Crystal 

Formal expressions for the quantum mechanical properties of a harmonic lattice are 
obtained just as easily as the classical properties—though their evaluation is rather 
more challenging. Because the system factorizes into N SHOs, the partition function 
is found by inserting the results from Section 23.10. 


Zqm = 


-huj{k) + kBTln{l — exp{—f}hu;{k))) 


(25.42) 


The energy of the system is just the sum of the energies of the individual modes, 
which can be confirmed by taking the negative logarithmic derivative of eq. (25.42) 
with respect to f3, as in eq. (23.11). 















Debye Approximation 301 


Uqm 


E 


k *- 


-nu{k) 


huj{k) 

exp(/3?ia;(fc)) — 1 


(25.43) 


Because the wave numbers are closely spaced, eq. (25.43) can be turned into an integral 
without introducing a significant error. 


Uqm 


L 

TT 



-nhj{k) + 


hjj{k) 

exp{f3huj(k)) — 1 


(25.44) 


For high temperatures, ksT ^ fnok for all modes, the average energy of every mode 
goes to ksT, and we recover the classical answer and the Law of Dulong and Petit. 

For low temperatures the situation is more complicated, because different modes 
have different frequencies. The contribution of each mode to the specific heat depends 
on the ratio when the ratio is small, the contribution to the specific heat 

is fcs, but when the ratio is large, the contribution becomes very small. As the 
temperature is lowered, the contributions to the specific heat from a larger fraction of 
the modes becomes negligible, and the total specific heat goes to zero. 

To calculate how the specific heat of the harmonic crystal goes to zero, we need the 
frequency spectrum found in Section 25.4. The formal solution for the specific heat is 
found by substituting eq. (25.33) into eq. (25.43) and differentiating with respect to 
temperature. This procedure is easily carried out numerically on a modern computer. 

On the other hand, it turns out to be both useful and enlightening to develop an 
approximation that allows us to investigate the low-temperature behavior analytically, 
as we will do in the following section on the Debye approximation. 


25.7 Debye Approximation 

We mentioned above that the Brillouin Zone is very simple in one dimension; it is just 
the region from k = —itla to it/a. In three dimensions it takes on more complicated 
forms that reflect the symmetry of the particular lattice: face-centered cubic, body- 
centered cubic, hexagonal, and so on. This makes it necessary to perform a three- 
dimensional integral over the Brillouin Zone. 

When the theory of the quantum mechanical harmonic lattice was first developed 
in the early part of the twentieth century, computers were non-existent; all calculations 
had to be accomplished by hand. This made it very important to simplify calculations 
as much as possible, and led to the approximation that we will discuss in this section. 

However, even today, when we can easily and rapidly evaluate eq. (25.44) and its 
generalization to three dimensions on a computer, there is still considerable value in 
simplifying the calculation as an aid to understanding its significance. Such simplifying 
approximations will also show the relationship of the harmonic crystal to black-body 
radiation, which might otherwise be much less obvious. 

The Debye approximation—named after its inventor, the Dutch physicist Peter 
Joseph William Debye (born Petrus Josephus Wilhelmus Debije, 1884-1966)—can 
be viewed as a kind of interpolation between the known high- and low-temperature 
regions. 








302 The Hsrmonic Solid 


For low temperatures, only low-energy modes will be excited significantly because 
the high-energy modes will be cut off by the exponential term exp^pHuj^k)) in the 
denominator of the integrand in eq. (25.44). Therefore, we only need to look at the low- 
frequency region of the energy spectrum. As shown in Section 25.4, the low-frequency 
spectrum is given by a simple linear relation. 


e{k) 


hujik) « hv 


(25.45) 


In this equation, v is the speed of sound, and we have generalized the result to an 
arbitrary number of dimensions. 

Eq. (25.45) is a very nice result because it is spherically symmetric. If the Brillouin 
Zone were also spherically symmetric, we could use that symmetry to reduce the three- 
dimensional integral for the energy to a one-dimensional integral. For this reason, 
Debye introduced an approximation that does exactly that. He approximated the 
energy spectrum by e(fc) = hv\k\ and the true shape of the Brillouin Zone by a sphere! 

It’s easy to see how Debye got away with this approximation at low temperatures. 
Only the low-energy modes made a significant contribution to the average energy, and 
eq. (25.45) was an excellent approximation for those modes. The high-energy modes 
that would be affected by true functional form of U!{k) and the shape of the Brillouin 
Zone did not contribute anyway, so that distortion did not matter. 

However, at high temperatures neither of these two arguments is valid. The high- 
energy modes are important, and they do not have the form given in eq. (25.45). 
Fortunately, a different argument comes into play to save the approximation. 

Recall that for high temperatures, when fc^T ^ tiuj{k), the energy of each mode is 
just given by the classical value of ksT. Since each mode contributes ksT, the energy 
spectrum is irrelevant; the only thing that matters is how many modes there are. For 
this reason, Debye fixed the size of his (approximate) spherical Brillouin Zone so that 
it had exactly 3N modes—the same as the true Brillouin Zone. 

The radius of the approximate Brillouin Zone is most easily found by going back 
into n-space, in which the points are distributed uniformly with a density equal to one. 
Each point in the positive octant {ux, riy, are all positive) represents three modes 
(one for each polarization direction). The total number of modes must equal 3iV. 

3 3 4 1 

3iV = - Airn^dn = —7rn|) = -irn^ (25.46) 

8 Jo 8 3 2 

The Debye radius in n-space is 


riD = 



(25.47) 


This also gives us results for the corresponding values of the Debye wave number and 
the Debye energy. 
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7T , 

f6N'' 

1/3 

(25.48) 

kD = 



) 


7T , 

f6N^ 

1/3 

(25.49) 

eb = 



) " 


Note that v in eq. (25.49) is the speed of sound, as in eq. (25.45). 

Since the ground-state energy plays no role in the thermodynamics of the harmonic 
solid, we will ignore it and write the energy in the Debye approximation for a three- 
dimensional harmonic crystal as a one-dimensional integral. 


U Debye 


Stt , , Huj{n) 

2 Jq exp{f3tuD{n)) - 1 


(25.50) 


At this point we will follow the same procedure for simplifying the integral that we 
used in Section 24.6 for black-body radiation. We define a dimensionless integration 
variable to transform the integral in eq. (25.50) to a more convenient form. 


X = (3fvjj{n) 


Trhv \ 
LkBTj ” 


(25.51) 


The upper limit of the integral in eq. (25.50) will become 

0D 


Xd = 


T 


where the Debye temperature, 0 d, is given by 

hv ( 67r^iV\ 


0_D = 


kB V 


- 


(25.52) 


(25.53) 


Using these substitutions, eq. (25.50) takes on the following form. 


3-77 r'd>D/r 

uDebye = ^ j J dx {exp{x) - 1)“ 

3L^{kBT)^ 


(25.54) 




dx x'^ (exp(a;) — 1) 


The great advantage of eq. (25.54) when Debye first derived it was that it could be 
evaluated numerically with pencil and paper (and a bit of effort). However, even today 
Debye’s equation remains very useful for gaining insights into the behavior of a crystal 
at both high and low temperatures, as we will investigate in the following subsections. 


25.7.1 Debye Approximation for T ^ ©£> 

At high temperatures we expect the energy to go to [/ « 3NkBT, because that was 
how we selected the size of the spherical Brillouin Zone. But it is always good to check 
to make sure that we did not make an error in the calculation. 
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For high temperatures, Qd/T 1, so that the upper limit of the integral in 
eq. (25.54) is small. For small values of x in the integrand, we can make the 
approximation 


(exp(x) — 1) ^ (1 + X — 1) ^ = 


(25.55) 


which is easy to integrate. Inserting this approximation into eq. (25.54), we find 


UDel)ye 


2'K'^h?V^ 


SL^ikeT)^ 1 fOo 


dx i 


2'K'^h?v^ 3 \ T 

3L^{kBTY 1 /hv /hTT^iVy/^ 1 
2 ' k '^} t ?‘ v ^ 3 yfcs \ y T 

L^{kBTYh^v^ /'6 tt^N\ 

2Tr'^h^v^ k^B V ) 

3NkBT 


(25.56) 


Since the Debye temperature was chosen to include 3N modes in the spherical 
approximate Brillouin Zone, the high-temperature expression for the energy is as 
expected, and the specific heat is SA:^, consistent with the Law of Dulong and Petit. 


25.7.2 Debye Approximation for T <^ &£> 

For low temperatures, Qd/T ^ 1, and the upper limit of eq. (25.54) is large. Since 
the integrand goes to zero as x^exp(—x), taking the upper limit to be infinite is a 
good approximation. 


Unebye ~ j (exp(x) - 1) ^ (25.57) 

The integral in eq. (25.57) should be familiar. It is exactly the same as the integral 
we found in eq. (24.29) during the analysis of black-body radiation. This is not a 
coincidence, as discussed below. 



15 


(25.58) 


Inserting the value of the integral into eq. (25.57), we find the low-temperature 
behavior of the energy of the harmonic lattice in closed form. 
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u, 


Debye 




L 


2'K^h^V^ 
TT^ 


15 


dkBTY 


lOh^v^ 


The energy per unit volume is then given by 


[/ 

V 


= u = 



(25.59) 


(25.60) 


The energy of the harmonic crystal at low temperatures, for which the Debye 
approximation is accurate, is very similar to the energy of black-body radiation. Recall 
eq. (24.30) from Chapter 24 for the energy per unit volume of black-body radiation. 


U 

V 


= u = 



(25.61) 


The form of eqs. (25.60) and (25.61) is the same, because for both black-body 
radiation and the low-temperature harmonic crystal, the energy spectrum is linear in 
the wave number: lo = vk for the harmonic solid, and uj = ck for black-body radiation 
(light). 

There are only two differences between eq. (25.60) for the low-temperature behavior 
of the harmonic solid and eq. (24.30) for black-body radiation. The expression for the 
harmonic crystal contains the speed of sound v, instead of the speed of light c, and 
the factor in the denominator is 10 instead of 15 because there are three polarizations 
for sound, but only two for light. 

From eq. (25.60) we see that the energy of a harmonic crystal is proportional to 
T^, which is analogous to the factor in the Stefan-Boltzmann Law, eq. (24.31). 
Taking a derivative of the energy with respect to temperature, we see that the specific 
heat is proportional to D, which is observed to be the correct behavior of insulating 
crystals at low temperatures. 


The specific heats of metallic crystals, on the other hand, are linear in the tem¬ 
perature. The origin of this difference between conductors and insulators will be 
discussed in Chapter 28. 


25.7.3 Summary of the Debye Approximation 

There are three steps in deriving the Debye approximation. 

1. Replace the true energy spectrum with an approximate spectrum that is linear 
in the wave numberj, k, and spherically symmetric: e(fc) = hv\k\. 

2. Replace the true Brillouin Zone by a spherically symmetric Brillouin Zone. 
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3. Choose the size of the spherically symmetric Brillouin Zone so that it contains 
exactly N fc-values and 3N modes. 

The Debye approximation is excellent at both high and low temperatures. Do not 
be fooled, however: in between, it is only mediocre. The qualitative behavior of the 
contributions to the specific heat from lattice vibrations in real crystals is monotoni- 
cally increasing with temperature, starting from T^-behavior at low temperatures and 
going to the constant value Sfeg (Law of Dulong and Petit) at high temperatures. The 
detailed form of the function depends on details of the true energy spectrum and the 
shape of the true Brillouin Zone. Fortunately, the full, three-dimensional integrals are 
easy to do on a computer, so that we need not rely on the Debye approximation as 
much as researchers did in the middle of the last century. 


25.8 Problems 


Problem 25.1 

The harmonic crystal 


1. Show that for an harmonic crystal the integral 



is exactly equal to the zero-point energy of the solid. 

2. Interpret the result graphically. It might help to relate this result to an earlier 
assignment on the properties of quantum SHOs. 

Problem 25.2 

The harmonic crystal with alternating spring constants 

Most crystals contain more than one kind of atom. This leads to both quantitative 
and qualitative differences in the vibration spectrum from the results derived in class. 
These differences occur whenever the periodicity is altered, whether by changing the 
masses of the atoms or changing the spring constants. 

As an example of the kinds of differences that can arise when the periodicity changes, 
consider the problem of a one-dimensional lattice with alternating spring constants 
Ki and K 2 . For simplicity, you can assume that all the masses are equal to m. 

1. If the distance between atoms is a, the periodicity is 2a. What is the size of the 
Brillouin Zone? 

2. Calculate the vibration spectrum of the lattice. You might find it more convenient 
to write down equations of motion for the two sublattices. 

3. Sketch the vibration spectrum in the Brillouin Zone based on your calculations. 
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4. Sometimes the vibration spectrum of a crystal with more than one kind of atom 
is approximated by a combination of an Einstein model and a Debye model. 
Explain why this might make sense and, give appropriate parameters for such 
a description in the present case. In answering this question, consider the high- 
and low-temperature behavior of the specific heat of this model. 


Problem 25.3 

The harmonic crystal with alternating masses 

Most crystals contain more than one kind of atom. This leads to both quantitative 
and qualitative differences in the vibration spectrum from the results derived in class. 
These differences occur whenever the periodicity is altered, whether by changing the 
masses of the atoms or changing the spring constants. 

As an example of the kinds of differences that can arise when the periodicity changes, 
consider the problem of a one-dimensional lattice with alternating masses mi and m 2 . 

1. If the distance between atoms is a, the periodicity is 2a. What is the size of the 
Brillouin Zone? 

2. Calculate the vibration spectrum of the lattice. You might find it more convenient 
to write down equations of motion for the two sublattices. 

3. Sketch the vibration spectrum in the Brillouin Zone based on your calculations. 
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Ideal Quantum Gases 


Out of perfection nothing can be made. Every process involves breaking 
something up. 

Joseph Campbell 


This chapter discusses the theory of ideal quantum gases. It develops general equations 
that will be valid for gases that consist of either bosons or fermions. Since the 
consequences of these equations are both non-trivial and very different for the two 
cases of bosons and fermions, the details of each of these cases will be dealt with in 
subsequent chapters. 

This chapter will also discuss the interesting case of a quantum gas of distin¬ 
guishable particles. Although all atoms are either bosons or fermions, and therefore 
indistinguishable, there are nevertheless real systems that are composed of distinguish¬ 
able particles. In particular, colloidal particles can each be composed of about 10® 
molecules. The number of molecules in each particle will vary, as will the arrangement 
and number of each type of molecule. In short, while the particles in a colloid might be 
similar in size and composition, they do not have identical properties and they are not 
indistinguishable. While the properties of colloids can usually be obtained accurately 
using classical statistical mechanics, it is also of interest to see how quantum mechanics 
might affect the results. 

We begin in the next section by discussing many-particle quantum states, which 
are needed to describe a macroscopic quantum system. 


26.1 Single-Particle Quantum States 

As is the case for most of statistical mechanics, we would like to put the theory of 
ideal quantum gases into a form that allows us to factorize the partition function. For 
ideal gases, that points to building up the macroscopic state in terms of single-particle 
states, which will then lead to such a factorization. 

Consider a quantum ideal gas in a box of volume V. For simplicity, we will assume 
that the box is cubic, with length L, so that V = L^. The sides of the box will be 
assumed to be impenetrable, also for simplicity. 
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For quantum ideal gases, we can write the Hamiltonian of a single particle of mass 
m as 


2m 2m 

where 


(26.1) 


(26.2) 


The eigenfunctions of eq. (26.1) can be written in a form that is very similar to 
that of the electric field in eq. (24.5) for the black-body problem, but is even simpler 
because the wave function -tp is a, scalar, while the electric field is a vector. 





sin (kxx) sin (kyy) sin (kzz) 


(26.3) 


(The derivation of the normalization constant is left to the reader.) 

The wave numbers in eq. (26.3) are determined by the condition that the wave 
function vanishes at the boundaries. 

fc. = ^ (26.4) 

fc, = ^ (26.5) 

fc. = ^ (26.6) 

If the values of n^, Uy, and riz are integers, the boundary conditions are fulfilled. 
Since the overall sign of the wave function is unimportant, negative values of these 
integers do not represent distinct states; only non-zero, positive integers represent 
physically distinct states. The wave equation can also be written in terms of the 
values of n = {n^,, n^, riz}. 


• 1 


1 sin 1 


1 sin 1 


V L3 1 

1 L 


1 L y) 


1 L V 


The energy eigenvalues for a single-particle state are easily found. 


2m 




- ^ + fcy + kl) - 


2toL2 


{nl + nl + nl) = 


(26.7) 


(26.8) 

(26.9) 


(The symbol e will be reserved for the energies of single-particle states, while E will 
be used for the energy of an iV-particle state.) 

If we define 


k — k^ ky -\- k^ 


( 26 . 10 ) 
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and 


2 2 , 2,2 

n = + Uy + 

we can express eq. (26.9) more compactly. 




hh 


2m 2mL‘^ 


(26.11) 


(26.12) 


Eq. (26.9) gives the single-particle energy eigenvalues for a particle in a three- 
dimensional, cubic box with sides of length L. This is the case on which we will 
concentrate. However, since we can preserve generality with little cost, I will replace 
the quantum number vector n with a general quantum number a for most of the rest 
of this chapter. This means that the equations in this chapter will be valid for any 
system of non-interacting particles, even if those particles are moving in an external 
potential. 


The energy eigenstates in eq. (26.9) are often called ‘orbitals’ for historical reasons; 
the early work on fermions concentrated on electrons orbiting around a nucleus, 
which led to the use of the term. It might seem strange to see the term ‘orbitals’ 
when nothing is orbiting, but tradition is tradition. I will refer to them as single¬ 
particle states, which is less compact, but also (I believe) less confusing. 


26.2 Density of Single-Particle States 

Just as we needed the photon density of states to calculate the properties of black 
bodies, we will need the single-particle density of states to calculate the properties of 
a system of particles. 

The density of states in n-space is easy to find. Since there is one state at every 
point with integer components, the density of states is 1. To calculate the density of 
states as a function of energy, we make a transformation using a Dirac delta function, 
following the methods developed in Chapter 5. 


D{e) 


POO poo POO 

/ drix / duy / dnzS{e-eft) 
Jo Jo Jo 


Aitn^dnS 



2mL2 





ei/2 


(26.13) 


V 


2m 

W 
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In the third line of eq. (26.13) we introduced a new variable of integration. 


X = 


\2mL2 ) 


(26.14) 


In the fourth line of eq. (26.13) we replaced by the volume of the system, V = L^. 

Eq. (26.13) is valid for non-interacting fermions, bosons, or distinguishable particles 
(without spin). It will play a central role in Chapters 27, 28, and 29. 


26.3 Many-Particle Quantum States 

An iV-particle wave function can be constructed from eq. (26.3) by multiplying single¬ 
particle wave functions. 


N 

= Y['4’aj{rj) (26.15) 

In this equation, aj represents the quantum number(s) that describe the wave function 
for the j-th particle. 

Eq. (26.15) is a correct wave function for distinguishable particles, and we will 
return to it in Section 26.9. However, for indistinguishable particles, the wave function 
must be symmetrized for Bose-Einstein statistics^ (bosons) or anti-symmetrized for 
Fermi-Dirac statistics^ (fermions). 

The wave function for bosons can be written as a sum over all permutations, P, 
of the assignments of particles to wave functions in the expression in eq. (26.15). 


= X be 


N 




i=i 


(26.16) 


The normalization constant, Xbe, can be calculated, but does not play a significant 
role and will be left to the interested reader. 

The wave function for fermions can be represented by a similar expression if we 
include a factor of (—1)^, in which the exponent denotes the number of two-particle 
exchanges needed to construct the permutation P. 




Af,5^(-l)^P 

p 


N 


1=1 


(26.17) 


Xpo is the normalization constant for Fermi-Dirac statistics, which will also be left 
to the reader. 


^Satyendranath Bose (1894-1974), Indian physicist, and Albert Einstein (1879-1956), German, Swiss, 
Austrian, and American physicist, Nobel Prize 1921. 

^Enrico Fermi (1901-1954), Italian physicist, Nobel Prize 1938, and Paul Adrien Maurice Dirac (1902- 
1984), British physicist, Nobel Prize 1933. 
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Note that the many-particle wave function for fermions vanishes if two or more 
particles are in the same single particle state (oi = aj for i ^ j). Each single-particle 
state can either be empty or contain one particle. There is no such restriction for 
bosons, and any number of bosons can occupy a single-particle state. 

The Hamiltonian of an A^-particle system of independent particles is just 

N 

= (26.18) 
1=1 

where Hj is the single-particle Hamiltonian for the j-th particle. 

The iV-particle eigenvalue equation is then 


N 

Hn '4^n = EiIjn = ^ tpN (26.19) 

1=1 

for bosons, fermions, or distinguishable particles. 

To construct the Boltzmann factor we need to apply the operator exp{—(3Hpf) to 
the wave function. Fortunately, this is easy. Simply expand the exponential in a series 
in powers of i?Ar, apply the eigenvalue equation, eq. (26.19), and sum the power series 
in E. 


exp(—/3iJAr)'0Ar = exp{—l3E)'il;N = exp 


tPN 


(26.20) 


26.4 Quantum Canonical Ensemble 

In principle, we could go to the canonical ensemble and sum the Boltzmann factor in 
eq. (26.20) over all IV-particle states, just as we did for black bodies in Chapter 24 
and harmonic solids in Chapter 25. Unfortunately, if we try to do this the canonical 
partition function does not factor. The difficulty has to do with enumerating the states. 
If the total number of particles is fixed, we can only put a particle in one state if we 
simultaneously take it out of some other state. We did not run into this problem 
for either black bodies or harmonic solids because neither photons nor phonons are 
conserved. 

To be able to sum freely over the number of particles in each single-particle state— 
and thereby factorize the partition function—we need to introduce a reservoir that can 
exchange particles with the system of interest. This brings us to the grand canonical 
ensemble, which is the subject of the next section. 

26.5 Grand Canonical Ensemble 

The grand canonical partition function was introduced for classical statistical mechan¬ 
ics in Sections 20.1 and 20.2. The equations describing the quantum grand canonical 
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ensemble are the same as those for the classical case, as long as we now interpret 
n(i?, N) to mean the degeneracy of an iV-particle quantum state with energy E. 

Eq. (20.9) gives the quantum mecanical grand canonical probability distribution 
for E and N. 

P{E, N) = ^n{E, V, N) exp [-/3E + PfiN] (26.21) 

Jit 

and eq. (20.10) gives us the quantum grand canonical partition function. 

OO 

^=EE n(£;,E, TV) exp [-/?£;+ /3/riV] (26.22) 

Af=0 E 

The sum over E in eq. (26.22) denotes the sum over all energy eigenstates of the 
iV-particle system. It replaces the integral over the continuum of energies in the 
classical case that was used in eq. (20.10). 

For a quantum system of non-interacting particles, the A'^-particle energies are 
given by 


N 


E = Ee. 

1=1 


(26.23) 


so that 

AT N 

exp(-/3E-h/rTV) = exp(-/3 ^(cq,^ -^)) = exp(-/3(e„^ - m))) (26.24) 

i=i 1=1 

We can also rewrite eq. (26.22) in terms of sums over the eigenstates, thereby 
eliminating the degeneracy factor n(iT, E, N). 

OO N 

Z = E E n exp(-/3(ea, - ft))) (26.25) 

N=0 {aj} 1=1 


26.6 A New Notation for Energy Levels 

To simplify the expression for the grand canonical partition function, it is useful to 
change the notation. So far, we have been describing an iV-particle state by listing 
the set of single-particle quantum numbers that specify that state. If more than one 
particle were in a particular single-particle state, the corresponding quantum number 
would occur more than once in the list. 

An alternative representation would be to specify for each state the number of 
times it appears in the list, which we will call the occupation number. 

Perhaps an example would help illustrate the two representations. In the following 
table I have indicated each state by a line and indicated the number of times it occurs 
by an ‘x’. 
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Table 26.1 Representations of energy levels. 


quantum number 
a 

Energy levels 

Occupation 
number, ria 

6 

- 

0 

5 

X 

1 

4 

XXX 

3 

3 

- 

0 

2 

X X 

2 

1 

X 

1 


To represent the data in Table 26.1, we could have a list of quantum numbers, 

Wj} = {cki, a2,02, Q!4, a 4 , 04 , Os} (26.26) 

or, equivalently, a list of occupation numbers for the energy levels, 

{n„} = {l,2,0,3,l,0} (26.27) 

Both representations contain exactly the same information. However, the representa¬ 
tion in terms of occupation numbers turns out to be more useful for the evaluation of 
the grand canonical partition function. 

At this point we shall make another change in notation, and indicate the energy 
levels by their energies instead of their quantum numbers. The notation ignores any 
possible degeneracy of the energy levels, although such degeneracy is important and 
must be included in all calculations. 


Denoting the energy eigenstates by their energy alone is, of course, not quite 
correct. Since energy levels can be (and often are) degenerate, this notation can 
be misleading. I really cannot defend it on logical grounds, but it does make the 
notation much more concise. It is also extremely common in the literature, so 
that you will have to come to terms with it sometime. Tradition is tradition. The 
important thing to keep in mind is that even though the sum is written as being 
over the energy levels, e, it is really over the energy eigenstates. 


The occupation of the state e is now denoted by n^. Note that for fermions, the 
only allowed values of are 0 and 1. For bosons, can be any non-negative integer. 

With this dubious change to denoting energy levels by their energy e, the expression 
for the grand canonical partition function becomes 

oo 

J|exp[-/3(e-^)n,] 

N—0 {rie} e 


(26.28) 
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where the sum over is understood to included only allowed values of rie. 

Now note that the sum over {rij} is limited to those cases in which ^ but 

the result is then summed over all values of N. We can simplify this double sum by 
removing the sum over N and removing the limit on the sum over {n^}. 

Z = ^ J^exp[-/3(e - fj.)n^] (26.29) 

The limitation to allowed values of is, of course, still in effect. 

Eq. (26.29) can be greatly simplified by exchanging sums and products in much the 
same way as we exchanged integrals and products for classical statistical mechanics 
in Section 19.12. Exchanging the sum and product is the key step that leads to the 
factorization of the grand canonical partition function. This trick is just as important 
in quantum statistical mechanics as it is in classical statistical mechanics, and so 
deserves its own section. 


26.7 Exchanging Sums and Products 

In eq. (26.25) we found a sum of products of the form He ~ M)’^e]i 

which can be transformed into a product of sums. This is another application of what 
we have called the best trick in statistical mechanics. 


^ Yl exp[-/3(e - pL)n ,], = ^ ^ ... 

{nj e "ei n^2 


= E' 


-/3(ei-/i)ne 


E' 


-0{e2-p)ne. 


= n E 

€ ne 


(26.30) 


As in Section 19.12, where we discussed the classical version of this trick, care must 
be taken with the indices of the sums on each side of the equality. The sum on the 
left side of eq. (26.30) is over the entire set {n^} for the 7V-particle system, while the 
sum in the last line on the right is only over the values taken on by the single-particle 
quantum number n^. 


26.8 Grand Canonical Partition Function for Independent Particles 

Inserting the equality in eq. (26.30) into eq. (26.25), we find the following expression 
for the grand canonical partition function. 

Z = (26.31) 

e rig 

The evaluation of the grand canonical partition function has been reduced to the 
product of sums over the number particles occupying each of the states. Each of these 
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sums will turn out to be quite easy to evaluate for ideal gases, making the transition 
to the grand canonical ensemble definitely worth the effort. 

The logarithm of the grand canonical partition function gives us the Legendre 
transform of the energy with respect to temperature and chemical potential, just as 
it did in the case of classical statistical mechanics. 

InZ =-/3[/[T,/i] (26.32) 

where 

f/[T, fi\ = U -TS - fiN (26.33) 

Once the grand canonical potential has been calculated, all thermodynamic informa¬ 
tion can be obtained by the methods discussed in Part 2. 

In particular, if the system is extensive, which is the case in most problems of 
interest, Euler’s equation tells us that 

lnZ = -/3UIT, = /3PV (26.34) 

This provides a good method of calculating the pressure in a quantum gas. 

Before we continue with the derivation of the basic equations for bosons and 
fermions, we will make a slight detour in the next two sections to discuss the 
properties of distinguishable quantum particles. We will return to identical particles 
in Section 26.11. 

26.9 Distinguishable Quantum Particles 

The most common systems of distinguishable particles are colloids, in which the 
particles are usually composed of many atoms. Because of the large mass of colloidal 
particles, it is rarely necessary to treat them quantum mechanically. On the other 
hand, it is instructive to see how the quantum grand canonical partition function for 
distinguishable particles differs from that of fermions or bosons. 

The distinguishing feature of a many-particle wave function for distinguishable 
particles is that it is not generally symmetric or antisymmetric. This means that any 
wave function of the form given in eq. (26.15) is valid. This equation is repeated here 
for convenience, with the notation modified to denote the product over single-particle 
states as a product over values of e. 


N 

'fpN (26.35) 

1=1 

Since the particles are distinguishable and can move between the system and the 
reservoir, we must include a factor giving the number of ways we can have N particles 
in the system and the remaining Nji particles in the reservoir. Letting the total number 
of particles be Nt = N + Nji, the factor is 


Nj^l 

NINrI 
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(26.36) 


This is exactly the same factor that we introduced in Chapter 4 for a classical ideal 
gas of distinguishable particles; the combinatorics are identical in the two cases. 

We must also avoid double counting when two or more particles are in the same 
single-particle state. This leads to a factor of 


Nl 

U.riel 


(26.37) 


If we include both of these factors, and note that the factor of NrI and NrI are 
not included in the entropy—or the grand canonical partition function—we find an 
expression for Z of the following form. 


^=nE 


1 

nj 


exp[-/3(e - 




ri)]) 


(26.38) 


If we take the logarithm of the partition function, this expression simplifies. 


In Z = ^ exp[—/3(e — fi)] = f3PV 


(26.39) 


The last equality is, of course, valid only in the case that the system is extensive. 


26.10 Sneaky Derivation of PV = NkBT 

The form of eq. (26.39) has a curious consequence. If we calculate the predicted value 
of N from the grand canonical partition function (see eq. (26.42)), we find 

{N) = keT^lnZ 

= ksT E ^ exp[-/3(e - ^l)] 

= exp[—/3(e — /r)] = In Z = f]PV (26.40) 

g 

where the last equality comes from eq. (26.39), and is valid for extensive systems. 
Eq. (26.40) is, of course, equivalent to 

PV = NkeT (26.41) 

if the fluctuations of N are neglected. 

This derivation of the ideal gas law depends only on distinguishability, extensivity, 
and the total wave function’s being a product of the single-particle eigenstates. 
Extensivity is necessary to eliminate, for example, a system in a gravitational field. 
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The most interesting feature about this derivation is that it is fully quantum 
mechanical. The ideal gas law does not depend on taking the classical limit! 

26.11 Equations for U = (E) and (N) 

In this section we return to calculations of the properties of bosons and fermions. 
Eq. (26.31) for the grand canonical partition function is, in principle, sufficient to 
perform such calculations. However, it turns out not to be the most efficient method 
for obtaining most properties of fermions and bosons. In this section we will derive 
the equations for the energy and number of particles, which will be the basis for the 
methods discussed in the rest of the chapter. 

Beginning with the formal expression for the grand canonical partition function in 
eq. (26.31), 


Z = J|^exp[-/3(e-/r)ne] (26.42) 

e rig 


we can easily find an expression for the average number of particles by taking a 
logarithmic derivative with respect to fi. 


^ In Z = ^ In ^ exp[-/3(e - ^l)n,] 


d^jL 


E 


Sn, exp[-/3(e - 

exp[-/3(e - /r)n,] 


= 

e 

= p{N) 


(26.43) 


The quantity (rie) clearly represents the average number of particles in the state 
labeled e. 


En,'»eexp[-/3(e-/r)rte] 
Ene exp[-/3(e - n)n^] 


(26.44) 


The equation for the average number of particles as a derivative of the grand 
canonical partition function is, of course, related to the corresponding thermodynamic 
identity in terms of the Legendre transform of the energy with respect to T and fi. 

Id Id d(PV) 

The average number of particles is then 


w = E(^^) 


( 26 . 46 ) 
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We will leave the proof of the corresponding equation for the energy to the reader. 


t/=(i?) = ^e(n,) 


(26.47) 


e 


Eqs. (26.46) and (26.47) turn out to be central to the calculation of the properties 
of fermions and bosons. Since we are going to use them frequently in the next two 
chapters, we will need to evaluate {rif} for both fermions and bosons, which will be 
done in the next two sections. 

26.12 (rie) for bosons 

For bosons, there are an infinite number of possibilities for the occupation of a single¬ 
particle state. This makes the evaluation of eq. (26.44) only slightly more difficult than 
the corresponding sum for fermions, since we can carry out the sum over all positive 
integers. 


OO 



(26.48) 


The numerator of eq. (26.44) for bosons can be obtained by differentiating 
eq. (26.48) with respect to (3. 


OO 


ne exp[-/3(e - n)n^] = exp[-/3(e - /r)] (1 - exp[-/3(e - ^)]) ^ (26.49) 


The average occupation number is then given by the ratio of these quantities, 
according to eq. (26.44). 


^ exp[-/3(e-/i)](l-exp[-/3(e-^)]) ^ 

(1-exp[-/3(e-/r)])"^ 

= (exp[/3(e -/i)] - 1)"^ 


(26.50) 


Note the strong similarity between the expression for the occupation number for 
bosons and the average number of excited particles in a simple harmonic oscillator, 
which we derived in eq. (23.72) in Section 23.10. They are all manifestations of the same 
mathematical structure, which we first encountered in the solution to the quantum 
simple harmonic oscillator. 

26.13 (rie) for fermions 

Since there are only two possibilities for the occupation of a single-particle state 
by fermions, the evaluation of eq. (26.44) is quite easy. The denominator and the 
numerator are calculated first. 
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exp[-/3(e - /i)ne] = 1 + exp[-/3(e - ^l)] (26.51) 

ne— 0 

1 

rig exp[—/3(e — /J.)ne] = 0 + exp[—/3(e — p)] (26.52) 

rie—0 

The average occupation number is then given by the ratio of these quantities, according 
to eq. (26.44). 


Note the strong similarity between the expression for the occupation number for 
fermions and the average number of excited particles in a two-level system, which we 
derived in eq. (23.56) in Section 23.9. 

26.14 Summary of Equations for Fermions and Bosons 

We can summarize the equations for (rig) by writing 

(rig) = (exp[/3(e - ^J.)] ± 1)“^ (26.54) 

where the upper (plus) sign refers to fermions and the lower (minus) sign refers to 
bosons. 

Eqs. (26.46) and (26.47) can then be written compactly as 

(iV) =y](exp[/3(e-/i)]±l)-i (26.55) 

e 

C/= y^e(exp[/3(e-/r)] ± 1)"^ (26.56) 


For the rest of the discussion in this chapter, as well as the following chapters on 
fermions and bosons, we will write N for {N). While this is not really correct, 
and we should keep the distinction between N and (N) in mind, it is tiresome to 
constantly write the brackets. 


We can express the grand canonical partition function for fermions by inserting 
eq. (26.51) in eq. (26.42). 


2 = Y[{1 + exp[-/3(e - /i)]) 


(26.57) 
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We can express the grand canonical partition function for bosons by inserting 
eq. (26.48) in eq. (26.42). 


Z = ]^(l-exp[-/3(e-/r)]) ^ (26.58) 

€ 

Finally, we can express the logarithm of the grand canonical partition function in 
a compact form for both cases. 

In Z = ± ^ In (1 ± exp[—/3(e — ft)]) = PPV (26.59) 

e 

The upper signs refer to fermions, and the lower signs to bosons. The last equality is 
valid only when the system is extensive, but that will be true of most of the cases we 
will consider. 


Warning: Do not forget that sums over e must count every eigenstate, not just every 
energy level. 


If we use the density of states that we calculated in Section 26.2, we can write the 
logarithm of the grand canonical partition function in terms of an integral. 


POO 

lnZ = ± / D{e)ln{l ± exp[—f3{e — ft)]) de = PPV 
Jo 

(26.60) 

26.15 Integral Form of Equations for N and U 


In Section 26.2 we calculated the density of single-particle states, 
interacting particles in a box, with the result given in eq. (26.13). 

D(e), for non- 

poo 

N= _D(e) (exp[/3(e — ^)] zb 1) ^ de 

Jo 

(26.61) 

pOO 

U= D(e) (exp[/?(e — ^)] zb 1) ^ede 

Jo 

(26.62) 


For more general systems, which we will discuss later in this chapter, the density 
of states, D(e), will have a different structure, which might be considerably more 
complex. However, we will still be able to express eqs. (26.55) and (26.56) in integral 
form. 

The integral form of the equations can still be used if the energy spectrum is 
partially discrete, if we represent the discrete part of the spectrum with a delta function 
of the form X6{e — ei), where the constant X represents the degeneracy of the energy 
level at ei. 

One of the most interesting aspects of eqs. (26.61) and (26.62) is that 
they completely separate the quantum statistics (which enter as the factor 
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(exp[/3(e —/i)] ± 1)”^) and the effects of the Hamiltonian (which are reffected in 
the density of states, D{€)). We will spend considerable effort in understanding the 
properties of systems of non-interacting particles, which might seem to be very special 
cases. Nevertheless, the methods we develop will apply directly to much more general 
cases, as long as we are able to calculate the density of states, D{e). We will even be 
able to obtain very general results in the following chapters for the consequences of 
the various kinds of structure in D{e) that can arise. 


26.16 Basic Strategy for Fermions and Bosons 

The basic strategy for dealing with problems of fermions and bosons differs from 
that usually employed for other problems in quantum statistical mechanics, where 
the first task is almost always to evaluate the canonical partition function. Although 
it is possible to solve problems with fermions and bosons by evaluating the grand 
canonical partition function, it is almost always better to use eqs. (26.46) and (26.47) 
(or eqs. (26.61) and (26.62) ) for the average number of particles and the average 
energy. 

The first step in solving problems involving fermions and bosons is to use eq. (26.55) 
or eq. (26.61) to obtain iV as a function of T and fi that is N = N{T, jj). 

Next, note that we rarely know the value of the chemical potential, while we almost 
always do know the number of particles in the system. Furthermore, the number of 
particles is generally fixed during experiments, while the chemical potential is not. 
This leads to inverting the function N = N{T,fi) that we found from eq. (26.55) or 
eq. (26.61) to give us ^ = ^(T, N). 

Finally, we use eq. (26.56) or eq. (26.62) to find the energy as a function of T 
and N (the latter through the function /i = ^{T,N)) This gives us two equations of 
state that will probably answer any questions in which we are interested. If we need 
a complete fundamental relation, we can—at least in principle—find it by integrating 
the equations of state, as discussed in Chapter 13. A fundamental relation can be 
found from eq. (26.59), which also provides a quick way of calculating the pressure for 
extensive systems. 

The details of how these calculations can be carried out in practice will be discussed 
in Chapters 27 and 28. 


26.17 P = 2U/3V 

The equation in the title of this section is easy to derive for a three-dimensional, 
monatomic, classical, ideal gas or a quantum gas of distinguishable particles. We need 
only combine the ideal gas law. 


PV = NkeT 


(26.63) 
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with the equation for the energy, 

U = ^NksT (26.64) 

to get 

(26.65) 

The surprising thing about eq. (26.65) is that it is also true for ideal Fermi and 
Bose gases, even though eqs. (26.63) and eqs. (26.64) are not. 

The proof begins with the integral form of the equations for the grand canonical 
partition function from eq. (26.60), 

poo 

lnZ = ± / D{e)ln{l ± exp[—f3{e — fi)]) de = PPV (26.66) 

Jo 

and the average energy from eq. (26.62) 

poo 

U= T)(e) (exp[/3(e —/J,)] ± 1) ^ ede (26.67) 

Jo 

In both of these equations, the density of states is given by eq. (26.13), 

=I? 

where we have introduced a constant X to simplify the algebra. 

To derive eq. (26.65) we integrate the expression for InZ by parts, so that 

poo 

lii2 = ±X / In (1 ± exp[—/3(e —/i)]) de (26.69) 

Jo 

becomes 


(26.70) 


InZ = ±X 


-e3/2in(l±exp[-/3(e-p)]) 


J 0 




/*00 2 

J (1 ± exp[-/3(e - p)])"^ (tP exp[-/3(e - n)]) de 


The first term on the right in eq. (26.70) vanishes exactly. Comparing eq. (26.70) with 
eq. (26.67) leaves us with 


InZ = 


2XP 


J (exp[/3(e - ^)] ± 1) ^de=‘^U (26.71) 


Since InZ = PPV for an extensive system, this gives us eq. (26.65). 
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It is rather remarkable that eq. (26.65) holds for any form of statistics in either 
classical or quantum mechanics. The factor 2/3 does depend on the system being 
three-dimensional, but that proof will be left to the reader. 


26.18 Problems 


Problem 26.1 

Identities in the grand canonical ensemble 

We have seen that derivatives of partition functions can be used to express various 
quantities of interest. Derive identities for the following in terms of derivatives of the 
quantum-mechanical, grand canonical partition function. 

1. The average number of particles in the system, (N), in terms of a derivative with 
respect to fi. 

2. The average number of particles in the system, (N), in terms of a derivative with 
respect to the fugacity. 

3. The average energy as a function of (3 and /i in terms of derivatives. 


Problem 26.2 

More on the grand canonical ensemble 

We have previously derived a general expression for the grand canonical partition 
function for a system of independent identical particles 

.^ = P ^ exp [-/3(e - 

e rig 

where the product over e is actually the product over the single-particle eigenstates. 
Each eigenstate must be counted separately, even when several eigenstates have the 
same energy (also denoted by e). 

The sum over rig is only over allowed values of n^. 

For bosons, an arbitrary number of particles can be in the same state, so can 
take on any non-negative integer value, rig = {0, 1 , 2 ,...}. 

For fermions, no more than one particle can be in a given state, so rie can only take 
on the values 0 and 1, = {0,1}. 

Do each of the following for both fermions and bosons 

1. Carry out the sum in the grand canonical partition function explicitly. 

2. Using the result from the previous question, calculate the average number of 
particles (N) in terms of (3 and fi. 

3. Calculate the average number of particles in an eigenstate with energy e, denoted 
by (n^). 
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4. Express the average number of particles (N) in terms of a sum over (rij). 

5. Calculate the average energy in terms of a sum over (n^). 


Problem 26.3 

Fermion and boson fluctuations 

Prove that the fluctuations about the average number of particles in an eigenstate, 
(n), is given by 

52(n) = (n)(l±(n)) 

where one of the signs is for fermions and the other is for bosons. 


27 

Bose-Einstein Statistics 


The world is full of magical things patiently waiting for our wits to grow 
sharper. 

Bertrand Russell (1872-1970), philosopher and mathematician 


Perhaps the most startling property of systems governed by Bose-Einstein statistics 
is that they can exhibit a phase transition in the absence of interactions between the 
particles. In this chapter we will explore the behavior of an ideal gas of bosons and 
show how this unusual phase transition arises. 

27.1 Basic Equations for Bosons 

We begin with the Bose-Einstein equations for N and U, taken from eqs. (26.55) and 
(26.56). 


N = '^{ne) ='^{exp[P{e-p)]-l) ^ (27.1) 

e e 

U = Y^ e{nf) = ^ e (exp[/3(e - p)\ - 1)"^ (27.2) 

e e 

We are interested in understanding experiments in which the chemical potential p 
is unknown and the number of particles N is fixed. On the other hand, as discussed in 
Chapter 26, eqs. (27.1) and (27.2) were derived under the assumption that p is known. 
This leads us to begin calculations by finding N = N{T,p) from eq. (27.1), and then 
inverting the equation to obtain the chemical potential, p = p{T,N). After we have 
found p, we can use eq. (27.2) to calculate the energy and the specific heat for a fixed 
number of particles. 

27.2 (rie) for Bosons 

The occupation number for bosons was derived in Chapter 26 and given in eq. (26.50). 

fBE{f) = {nfj = (exp[/3(e - p)] - 1)"^ (27.3) 

Although the expressions for the occupation numbers for bosons and fermions only 
differ by a plus or minus sign in the denominator, the consequences of this small 
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difference are enormous. The first consequence of the minus sign in the denominator 
of eq. (27.3) for bosons is a limitation on the allowed values of the chemical potential. 
Since /B_E(e) gives the average number of particles in a single-particle state with energy 
e, it cannot be negative. 

(exp[/3(e - n)] - 1)"^ > 0 (27.4) 

This immediately requires that 

e > /i (27.5) 

for all values of e. In particular, if e = 0 is the lowest energy of a single-particle state, 
then 


^ < 0 (27.6) 

The chemical potential of a gas of bosons must be negative. 

Eq. (27.6) is almost a general result. It assumes that the lowest energy level is zero, 
which might not always be true. Because this can be a trap for the unwary, it is 
very important to know why fj, is algebraically less than the energy of the lowest 
single-particle state, instead of just remembering that it is less than zero. 


27.3 The Ideal Bose Gas 

For systems with a continuous energy spectrum, we expect to be able to change 
the sums in eqs. (27.1) and (27.2) into integrals and use eqs. (26.61) and (26.62) to 
calculate the properties of the system. For bosons, these equations take the following 
form. 


poo 

N= DBE{e){exp[P{e-de 

Jo 

(27.7) 

pOO 

u = 1 Dbe{^) e (exp[/3(e - //)] - 1)“^ de 

Jo 

(27.8) 


A particularly important case, which we will discuss in detail, is an ideal gas of 
bosons. We have already derived the single-particle density of states for particles in a 
box in eq. (26.13). 


DBEie) = ^ (27.9) 

Eq. (27.9) completes the equations we need to investigate the properties of free 
bosons. We will begin to calculate those properties by looking at the low-temperature 
behavior of the chemical potential in the next section. 
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27A Low-Temperature Behavior of n 

We can simplify the problem of finding the chemical potential from eq. (27.7) by 
making the integral dimensionless. To do this, we introduce a dimensionless variable 
X = /3e. 

poo T7- 

^ = 4^ [jfiJ e^^^(exp[/3(e-/r)]-l)”^de (27.10) 

= (1^) (exp(-/3/i) exp(a;) - 1)"^ dx 

It is convenient at this point to introduce the fugacity 

A = exp(/3^) (27.11) 

Note that because /r < 0, the fugacity must be less than 1, and its reciprocal, which 
enters into eq. (27.10), must be greater than 1. 

A"^ > 1 (27.12) 

In terms of the fugacity, the equation for N becomes 

^ ^ exp{x) - ly^ dx. (27.13) 

The form of eq. (27.13) is extremely important in understanding the properties of 
bosons. As the temperature is decreased, the factor of in front of the integral 
also decreases. Since the total number of particles is fixed, the dimensionless integral 
in eq. (27.13) must increase. The only parameter in the integral is the inverse fugacity, 
so it must vary in such a way as to increase the value of the integral. Since the inverse 
fugacity is in the denominator of the integrand, the only way to increase the value 
of the integral is to decrease the value of the inverse fugacity. However, because of 
eq. (27.12) we cannot decrease the value of the inverse fugacity below A“^ = 1. If the 
value of the dimensionless integral in eq. (27.13) diverged as A”^ —> 1, there would be 
no problem. However, although the integrand diverges at cc = 0 when A = 1, the value 
of the integral is finite. 

J (exp(x) — 1)”^ da: = = 1.306-\/7r = 2.315 (27.14) 


It is traditional to express the value of the integral in eq. (27.14) in terms of a 
C-function instead of simply 2.315. This is largely because the theory of bosons 
was developed before computers were available. At that time, instead of simply 
evaluating integrals numerically, great efforts were made to express them in terms 
of ‘known’ functions; that is, functions whose values had already been tabulated. 
Although retaining the (^-function is a bit of an anachronism, I have included it and 
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the factor of ^/tt in eq. (27.14) in the following equations for the sake of tradition 
(and comparison with other textbooks). 


The consequence of the dimensionless integral in eq. (27.14) having an upper bound 
is that the equation cannot be correct below a temperature Te that is determined by 
setting the integral equal to its maximum value. 

' (fcBT£)3/2l.306V^ (27.15) 

This equation can also be written as 

TV = 2.315^ ^ (kETsf'^ (27.16) 

The temperature Te is known as the Einstein temperature. 


ksTE 


2nh^\ / N 
m ) V2.612E J 


(27.17) 


For fixed N and temperatures less than Te, eq. (27.13) has no solution, even 
though you can certainly continue to cool a gas of bosons below Te- This apparent 
contradiction has an unusual origin. Even though the energy levels are very closely 
spaced, so that changing from a sum in eq. (27.1) to the integral in eq. (27.7) would 
not normally be expected to cause difficulties, eq. (27.7) is not valid for T < Te- 
In the next section we will show how to remove the contradiction by modifying 
eq. (27.7) to treat the lowest energy state explicitly. 


27.5 Bose-Einstein Condensation 

We can trace the strange behavior of bosons at low temperatures to the form of the 
boson occupation number in eq. (27.3). 

/B£;(e) = {n-e) = (exp[/3(e - /i)] - 1)"^ (27.18) 

The difficulty arises from the fact that the occupation of the lowest energy level, e = 0, 
has an occupation number 

fBE{0) = (no) = No = (exp[-/3/i] - I)"' (27.19) 

If we were to set ^ = 0, the occupation number would be infinite. This means that for 
very small but non-zero values of fi, the occupation number of the e = 0 state can be 
arbitrarily large. In fact, it can even contain all the bosons in the system! 

As will be justified in Section 27.8, the only modification we need to make 
in eq. (27.7) is to include the number of particles in the e = 0 state, which we 
denote as Nq. 
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•OO 


N = No+ / L>B£;(e) (exp[/3(e - ^)] - 1) ^ de 


(27.20) 


0 


For free bosons this becomes 



Above the temperature the occupation Nq of the e = 0 state is much smaller 
than the total number N of particles in the system, and may be neglected. Below Te, 
the value of Nq is comparable to N. That is, a significant fraction of the particles in 
the system are in the single-particle state with the lowest energy. The transition at Tg 
is called the Bose-Einstein condensation, because a significant fraction of the particles 
‘condense’ into the e = 0 state at temperatures below Te- 

27.6 Below the Einstein Temperature 

If we compare eq. (27.21) with eq. (27.13), which determines the Einstein temperature, 
we see that we can rewrite eq. (27.21) as 


3/2 



(27.22) 


We can then solve this equation for the occupation of the e = 0 state below Te- 



(27.23) 


Eq. (27.23) shows that Nq ^ N as T ^ 0; for zero temperature all particles are in the 
lowest single-particle energy state. 

Eq. (27.23) also shows that as T ^ Te, Nq 0. This result is, of course, only an 
approximation. What it means is that above Te, Nq <C N, so that it can be safely 
ignored. 

Note that as T ^ Te from below, Nq goes linearly to zero. 



(27.24) 


If we combine eq. (27.23) with eq. (27.19), we can find the temperature dependence 
of the chemical potential. 



3 / 2 ‘ 


Nq = [exp(-/3/r) - 1] ^ = N 1 - 


(27.25) 
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Since we know that (3^ is very small below Te, we can expand exp(—/3/r) in eq. (27.25) 
to find a very good approximation for 




ksT f T 

N \Te) 


(27.26) 


Because of the factor of 1/A^ in eq. (27.26), the chemical potential /i is extremely 
small below the Einstein temperature Te', if N is of the order of Avogadro’s number, 
|/i| is smaller than 

The approximation in eq. (27.26) also shows that as T —*■ Te, the value of p rises 
sharply. If T is close enough to the Einstein temperature, the approximation breaks 
down. However, this only happens when T is extremely close to Te- For all practical 
purposes, eq. (27.26) can be used for all temperatures below Te- 

Now that we have found the chemical potential for an ideal gas of bosons, we are 
in a position to calculate the energy and the specific heat, which we will do in the 
next section. 


27.7 Energy of an Ideal Gas of Bosons 

Given the chemical potential, we can calculate the energy of a gas of bosons from 
eq. (27.8). 

poo 

U = Uo+ DBE{e)e{exp[f3ie-^l)]-l)-\de (27.27) 

Jo 

In this equation. 


C/o = 0 (27.28) 

is the energy of a particle in the lowest energy level is e = 0. 

Introducing the dimensionless variable x = fJe, as we did in eq. (27.13), we can 
write an equation for U in terms of a dimensionless integral. 

C7 = ^ (^) {ksT)^^'^ J (A“^ exp(a;) - l) ^ dx (27.29) 

Below Te, eq. (27.29) simplifies because = 1. The dimensionless integral can 
be evaluated numerically. 

^ (exp(x) - 1)”^ dx = C r = 1.341 ^ (27.30) 

In this equation, r(-) indicates the gamma function and C(') again indicates the zeta 
function. 

Using eq. (27.30), eq. (27.29) becomes 

V = 1.7826jE (^n)’' 


(27.31) 
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Since we know from eq. (27.15) that 
^ = 2 - 315 ^ 


we find that the energy per particle has a simple form. 


U 

N 


/1.7826\ 
V 2.315 ) 



■ill 

ksT = .7700 



(27.32) 


(27.33) 


The specific heat at constant volume is found by differentiating with respect to 
temperature. 


cy = 1.925 ks 



(27.34) 


Notice that for T = Te, the specific heat takes on the value of 1.925 which 
is greater than the classical value of 1.5 ks that it takes on in the limit of high 
temperatures. 

For temperatures above Te, the specific heat of an ideal boson gas decreases 
monotonically to its asymptotic value of 1.5 ks- The derivation involves expanding 
the integral in eq. (27.8) as a function of /3/i. Since the expansion involves interesting 
mathematics, but little physics, I will refer the readers to the many textbooks that 
give the details. The fastest and easiest way to obtain the function today is to carry 
out the integral in eq. (27.8) numerically. 


27.8 What About the Second-Lowest Energy State? 

Since we have seen that the occupation of the lowest energy level has such a dramatic 
effect on the properties of a gas of bosons, it might be expected that the second-lowest 
energy state would also play a significant role. Oddly enough, it does not, even though 
the second-lowest energy state lies only slightly higher than the lowest energy state. 

The energy levels are given by eq. (26.7). Since the wave function in eq. (26.3) 
vanishes if Ux, riy, or Uz is zero, the lowest energy level, Eq, corresponds to n = (1,1,1). 
This gives us 


Co = £(1,1,1) 


2 ..^2 


h^Tr‘‘ . o „ h~Tt 


2mL2 


(27.35) 


[Remember the warning about the lowest energy level deviating from e = 0.] 

The next lowest energy level, ei, corresponds to n = (1,1, 2), or (1, 2,1), or (2,1,1) 
These three states each have the energy 
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Cl — £(1.1,2) 


h^TT^ 

2mL’^ 


(12 + 1 ^ + 22 ) =6 


ti^Tr^ 

2mL2 


The difference in energy between the two states is 


Ae 


h^TT^ 


(27.36) 


(27.37) 


The energy difference found in eq. (27.37) is extremely small. Suppose we consider 
a system consisting of ^He in a cubic container with T = 1 cm. The energy difference 
is only about 2.5 x 10“^® erg. Expressing this as a temperature, we have 

^ « 1.8 X a: (27.38) 

Kb 

Despite the fact that this energy difference is so small, it has a large effect on the 
occupation number of the state because it is substantially larger than the chemical 
potential. From eq. (27.26) we see that [3g « —1/A w —10“^^, so that the occupation 
number, (ni), of the second-lowest energy levels is 

(ni) = (exp(/3(Ae - g)) - 1)“^ (27.39) 

« (exp(/3Ae) - I)"' 

1 

JKe 

T 

1.8 X 10-14 A’ 

« (5.5 X 10^^ K-^)T 


where the last two lines comes from eq. (27.38). The occupation of the second lowest 
states is of the order of 10^^ of more, but this is still very small in comparison to the 
total number of particles, N « lO^^, or the number of particles in the lowest energy 
state below Te- Because the numbers of particles in each of the states above the 
lowest-energy state are so much smaller than the number in the lowest-energy state, 
it is completely sufficient to use the integral in eq. (27.20) to calculate their total 
contribution. 


27.9 The Pressure below T < Te 

Eq. (26.65) has an interesting consequence for the pressure of a boson gas 
below the Bose-Einstein condensation. If we insert eq. (26.65) into eq. (27.31), 
we obtain 


2 U 

P= 3^ = 0.2971 




(27.40) 
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This means that the pressure along any isotherm for a gas of bosons below Te is 
constant. It depends on the mass of the particles and the temperature, but nothing 
else. Another way of expressing this is to say that the isothermal compressibility of 
an ideal Bose gas is infinite below the Bose-Einstein transition. 

27.10 Transition Line in P-V Plot 

We can combine eq. (27.17) with eq. (27.40) to find the Bose-Einstein transition line 
in a P-V plot. 



(27.41) 


As eq. (27.41) shows, the pressure is proportional to the (5/3)-power of the number 
density along the transition line. 


27.11 Problems 


Problem 27.1 

Bosons in two dimensions 

Consider an ideal boson gas in two dimensions. The N particles in the gas each have 
mass m and are confined to a box of dimensions L x L. 

1. Calculate the density of states for the two-dimensional, ideal Bose gas. 

2. Calculate the Einstein temperature in two dimensions in terms of the given para¬ 
meters and a dimensionless integral. You do have to evaluate the dimensionless 
integral. 

Problem 27.2 

Bosons in four dimensions 

Consider an ideal boson gas in four dimensions. The N particles in the gas each have 
mass m and are confined to a box of dimensions L x L x L x L. 

1. Calculate the density of states for the four-dimensional, ideal Bose gas. 

2. Calculate the Einstein temperature in four dimensions in terms of the given 
parameters and a dimensionless integral. You do not have to evaluate the dimen¬ 
sionless integral. 

3. Below the Einstein temperature, calculate the occupation number of the lowest- 
energy, single-particle state as a function of temperture, in terms of the Einstein 
temperature and the total number of particles. 

4. Calculate the chemical potential below the Einstein temperature. 


Problem 27.3 

Energy of an ideal Bose gas in four dimensions 
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Again consider an ideal boson gas in four dimensions. The N particles in the gas 
each have mass m and are confined to a box of dimensions L x L x L x L. Calculate 
the energy and the specific heat as functions of temperature below the Einstein 
temperature. 


28 

Fermi—Dirac Statistics 


There are two possible outcomes: If the result confirms the hypothesis, then 
you’ve made a measurement. If the result is contrary to the hypothesis, then 
you’ve made a discovery. 

Enrico Fermi 


In the previous chapter we investigated the properties of bosons, based on the general 
equations for quantum gases derived in Chapter 26. The same equations—with a 
positive sign in the denominator—govern fermions. Nevertheless, the properties of 
fermions are dramatically different from those of bosons. In particular, fermions 
do not exhibit any phase transition that might correspond to the Bose-Einstein 
condensation. 

The most important fermions are electrons, and understanding the properties of 
electrons is central to understanding the properties of all materials. In this chapter we 
will study the ideal Fermi gas, which turns out to explain most of the properties of 
electrons in metals. In Chapter 29, we will see how Fermi-Dirac statistics also explains 
the basic features of insulators and semiconductors. 

We will begin in the next section with the fundamental equations for fermions. 

28.1 Basic Equations for Fermions 

We begin with the Fermi-Dirac equations for N and U, taken from eqs. (26.55) and 
(26.56). 


e e 

(28.1) 

U = e{nf) = ^ e (exp[/3(e - p)] + 1)"^ 

(28.2) 


e e 


We follow the same basic procedure as we did for bosons in the previous chap¬ 
ter by finding N = N{T,fi) from eq. (28.1), and then inverting the equation to 
obtain the chemical potential, /r = p(T, N). However, both the mathematical methods 
and the physical results for fermions are quite different than what they are for 
bosons. 
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28.2 The Fermi Function and the Fermi Energy 

The average occupation number, (rig), for fermions in an eigenstate with energy e was 
calculated in eq. (26.53). It is called the Fermi function, and it is written as 


fpoie) = (rie) = (exp[/3(e - ^)] + 1) ^ 


(28.3) 


To solve for the properties of a Fermi gas, it is extremely helpful to have a clear idea 
of the form of the Fermi function, which is shown for a moderately low temperature 
in Fig. 28.1. 

For lower temperatures (higher (3 = l/ksT), the Fermi function becomes steeper 
at e = p. The derivative of the Fermi function is given by 


= f'pa{e) = -/3exp[/3(e - /i)] (exp[/3(e - p)] + 1) ^ 


(28.4) 


Evaluating the derivative at e = ^ gives the maximum value of the slope. 



(28.5) 


It is clear from the form of the function that the width of the non-constant part of 
the function near e = is of the order of 2kBT. As the temperature is lowered, the 
Fermi function approaches a step function. At T = 0, the Fermi function is equal to 1 
for e < /i, 0 for e > /i, and 1/2 for e = ^. 

For most problems of interest, ksT /i, so that treating the Fermi function as a 
step function is a very good first approximation. Indeed, the Fermi function for most 
materials of common interest at room temperature is much closer to a step function 
than the curve shown in Fig. 28.1. 

As mentioned above, the total number of particles, N, is inevitably fixed in any 
system we will discuss, so that the chemical potential is a function of N and the 
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Fig. 28.1 The form of the Fermi function. The units of energy have been chosen such that /r = 1, 
and ft = W in this figure. 
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temperature T. However, as we will see later, the chemical potential is a rather weak 
function of temperature, so that its zero-temperature limit usually provides a very 
good approximation to its value at non-zero temperatures. This limit is so important 
that it has a name, the Fermi energy, and is denoted by eij-. 

ep = lim^F(T,N) (28.6) 

Because the Fermi function becomes a step function at T = 0, the Fermi energy 
must always lie between the energy of the highest occupied state and that of the 
lowest unoccupied state. In fact, as the problems in this section will show, it always 
lies exactly half way between the energies of the highest occupied state and the lowest 
unoccupied state. This gives a simple rule for finding the Fermi energy that greatly 
simplifies fermion calculations. 


The definition in eq. (28.6) agrees with that of Landau and Lifshitz in their classic 
book on statistical mechanics. Unfortunately, many textbooks give a different defini¬ 
tion (the location of the highest occupied state at T = 0), which only agrees with the 
Landau-Lifshitz definition when energy levels are quasi-continuous. The alternative 
definition loses the connection between the Fermi energy and the chemical potential 
for systems with a discrete energy spectrum, or for insulators and semiconductors. 
This is a serious handicap in solving problems. Using the definition of ep given in 
eq. (28.6) makes it much easier to understand the behavior of such systems. The 
rule of thumb that the Fermi energy lies exactly half way between energies of the 
highest occupied state and the lowest unoccupied state is easy to remember and 
use in calculations. Finding ep should always be the first step in solving a fermion 
problem. 


28.3 A Useful Identity 

The following identity is very useful in calculations with fermions. 

(exp[/3(e - n)] + 1)"^ = 1 - (exp[-/3(e - yt)] + 1)"^ (28.7) 

The left side of eq. (28.7) is, of course, the Fermi function, fpo- On the right side, the 
second term gives the deviation of the Fermi function from 1. 

It is easy to prove the identity in eq. (28.7), and I strongly recommend that you 
do so yourself. It might be easier to prove it again during an examination than to 
remember it. 


For energies more than a couple of factors of kpT above the chemical potential 
the Fermi function is very small, and can be approximated well by neglecting the ‘-I-1’ 
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in the denominator. 

Jfd = (exp[/3(e - p.)] + 1)“^ w exp[-/3(e - /i)] (28.8) 

For energies more than a couple of factors of ksT below the chemical potential /r, 
the Fermi function is close to 1, and the second term on the right side of eq. (28.7) 
can be approximated well by neglecting the ‘+1’ in its denominator. 

Jfd = 1 - (exp[-/3(e - ft)] + 1)"^ « 1 - exp[/3(e - p,)] (28.9) 

These simple approximations are the key to solving fermion problems with a 
discrete spectrum or a gap in a continuous spectrum. 

Up to this point we have been discussing the effects of Fermi statistics on the 
occupation number as a function of the energy of a given state. Now we will investigate 
how Fermi statistics affect the properties of systems with different distributions of 
energy levels. 

28.4 Systems with a Discrete Energy Spectrum 

We will first consider systems with a discrete spectrum, like that of the hydrogen 
atom. Much of the early work on the properties of fermions dealt with electronic 
behavior in atoms, and built on the exact quantum solution for the hydrogen 
atom to construct approximations in which electrons were taken to be subject 
to the field of the nucleus, but otherwise non-interacting. Since the electrons 
were orbiting the nucleus, the single-particle eigenstates were called orbitals—a 
name that has stuck, even for systems in which the electrons are not orbiting 
anything. 

Assume we have non-interacting fermions, each of which has the same energy 
spectrum composed of discrete single-particle eigenstates. Let the states be labeled by 
an index j, the energies of the states be denoted by e^, and the degeneracy of the j-th 
state by gj. 

Recall from Section 26.16 that the key to solving problems for fermions and bosons 
is to use the equations for N and U. Those equations were given in eqs. (26.46) and 
(26.47). Inserting the Fermi function in these equations, we find 

N = Y^gj (exp[/3(ej - g)] + 1)"^ (28.10) 

3 

and 

U = J2gjej (exp[/3(ej - g)] + 1)"^ (28.11) 

3 


It is so much more valuable to work out the consequences of eqs. (28.10) and (28.11) 
yourself than to see somebody else’s solutions, that I am not going to present 
examples at all—^just homework problems at the end of the chapter. 
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28.5 Systems with Continuous Energy Spectra 

For systems with a continuous energy spectrum, we can use the integral forms in 
eqs. (26.61) and (26.62) to calculate the properties of the system. For fermions, these 
equations take the following form. 


poo 

N= DFD{e){exp[f3{e-de 

Jo 

(28.12) 

poo 

u = 1 (exp[/3(e - fi)] + 1)“^ de 

Jo 

(28.13) 


The Fermi energy is given by the zero-temperature limit of the chemical potential, 
as shown in eq. (28.6). In that limit, the Fermi function becomes a step function, and 
N is determined by the equation 

N= [ DFD{e)de (28.14) 

Jo 

If we know the density of states, D{e), we can solve eq. (28.14) for the Fermi energy 
ep. In the following sections we will carry this out explicitly for an ideal Fermi gas. 

28.6 Ideal Fermi Gas 

The case of an ideal gas of fermions in three dimensions is extremely important. This 
might seem surprising, since electrons certainly interact through the Coulomb force, 
and that interaction might be expected to dominate the behavior in a macroscopic 
object. Fortunately, for reasons that go well beyond the scope of this book, electrons in 
a solid behave as if they were very nearly independent. You will understand why this 
is true when you take a course in many-body quantum mechanics. In the meantime, 
just regard it as a stroke of good luck that simplifies your life. 

We have already derived the single-particle density of states for particles in a box 
in eq. (26.13). To apply it to electrons we need only multiply it by 2 to take into 
account the spin of the electron, which takes on two eigenvalues. 

Dpoie) = ^ e'/" (28.15) 


28.7 Fermi Energy 

Since the Fermi energy will prove to be a good first approximation to the chemical 
potential, we will begin by calculating it from the density of states. 

In the limit of zero temperature, the Fermi function becomes a step function, and 
we use eq. (28.14) for the number of particles to find an equation for ep. 


N= [^DFD{e)de= [ 
Jo Jo 


EF y 

2tt'^ 



V 

27r^ 


/2m V/' 2 3/2 

J 3'^ 


(28.16) 
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This equation can then be inverted to find ep as a function of N. 

^ 2m \ / 


(28.17) 


Note that the Fermi energy depends on the two-thirds power of the number density 
N/V, and inversely on the particle mass. 

Because of the frequency with which the product (3ep appears in calculations, 
Fermi energy is often expressed in terms of the Fermi temperature, Tp. 


cp = kpTp 


(28.18) 


Pep 


Tp 

T 


(28.19) 


Fermi temperatures turn out to be remarkably high. Tp for most metals ranges 
from about 2 x lO'^K to 15 x 10^/F. For comparison, the temperature of the surface 
of the sun is only about 6 x 10^K. For experiments at room temperature (SOOTF), 
this means that T/Tp « 0.01, justifying the statement made in Section 28.2 that the 
Fermi function is very close to a step function for real systems of interest. Again for 
comparison. Fig. 28.1 was drawn for the case of T/Tp « 0.1. 

Once we have the Fermi energy, the total energy of the system at T = 0 can be 
obtained from eq. (28.13), using the step function that is the zero-temperature limit 
of the Fermi function. 


U = 



V 

DpD{e)ede = 



V 

27r^ 


/2mV/" 2 5/2 


(28.20) 


Combining eq. (28.16) with eq. (28.20), we find that the energy per particle takes 
on a particularly simple form. 


U 

N 



(28.21) 


28.8 Compressibility of Metals 

Metals are hard to compress. They also contain electrons that are fairly well described 
by the ideal gas of fermion we have been discussing. Perhaps surprisingly, these two 
facts are closely related. 

If we take the energy of an ideal gas of fermions in terms of the Fermi energy from 
eq. (28.21) and insert the expression for the Fermi energy from eq. (28.17), we find an 
expression for the energy as a function of the volume V. 


U 

N 


3 fP 
5 2m 


3n‘^N 


V 


2!?, 


(28.22) 
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Recalling the definition of the pressure as a derivative of the energy, 

dU 


P=- 


dV 


we find 


or 


p = -N~ = (^)v 

5 2m ^ dV 5 2m '' ^ \ ^ J 

2 ** (, 2 ) 2/3 


- 5/3 


P = 


5 2m 


) 


Recalling the definition of the compressibility from eq. (14.14), 


= =-l 

V \dPJ 


' dV 


T,N 


we find that 


3m 


- 4/3 


^n5/3 

N 


This can also be written in terms of the bulk modulus. 


K'j' 


dP\ 




dv J 


T,N 


3m 




V 


5/3 


(28.23) 


(28.24) 


(28.25) 


(28.26) 


(28.27) 


(28.28) 


Comparing eq. (28.28) with the expression for the Fermi energy in eq. (28.17) 


/37r2/V\"/^ 

^ V V J 


we find a surprisingly simple result. 


^^5 2 ^ 

kt 3 3 R 


cf 


(28.29) 


(28.30) 


The numerical predictions of eq. (28.30) turn out to be within roughly a factor 
of 2 of the experimental results for metals, even though the model ignores the lattice 
structure entirely. This is remarkably good agreement for a very simple model of a 
metal. It shows that the quantum effects of Fermi-Einstein statistics are responsible 
for a major part of the bulk modulus of metals. 


28.9 Sommerfeld Expansion 

While we have obtained important information about Fermi gases from the zero- 
temperature limit of the Fermi function in the previous section, the behavior at 
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non-zero temperatures is even more interesting. The difficulty in doing calculations for 
T yf 0 is that the temperature dependence of the integrals in eqs. (28.12) and (28.13) 
is not trivial to extract. 

The problem was solved by the German physicist Arnold Sommerfeld (1868-1951). 
His solution is known today as the Sommerfeld expansion, and it is the topic of the 
current section. The results obtained from the Sommerfeld expansion are essential for 
understanding the properties of metals. 

First note that both eq. (28.12) and eq. (28.13) can be written in the form, 


pOO pOO 

</'(e)/F(e)de = / ^(e) (exp[/3(e -/r)]-k 1)“^ de (28.31) 

Jo Jo 

where ^(e) = DpD{e) to calculate N, and (j){e) = eDpoi^) to calculate U. 

Break the integral in eq. (28.31) into two parts, and use the identity in eq. (28.7) 
to rewrite the integrand for e < /r. 

^=[ <('(e)(exp[/3(e-M)] + l)”^* 

poo 

+ <('(e)(exp[/3(e-/r)]-kl)"^de 

Jfl 

= [ <('(e) 1 - (exp[-/3(e-/r)]-k 1)"^ de 
Jo 

pOO 

+ / HO (exp[/3(e - fi)] + 1)"^ 

= f HOde- f </>(e)(exp[-/3(e-/i)]-kl)"^de 
Jo Jo 


+ <('(e)(exp[/3(e-/r)]-kl) 


-1 


(28.32) 


In this equation, the first integral represents the contributions of a step function, while 
the second and third integrals represent the deviations of the Fermi function from the 
step function. 

The next step is to substitute the dimensionless integration variable z = —/3(e — /i) 
in the second integral, and z = /3(e — p,) in the third integral. 


rn rpp 

X= / (j>{e)de — P~^ / — f3~^z) {e^ + 1)~^ dz 

Jo Jo 

poo 

-k/3”^ / + /J-H) + 1)~^ dz (28.33) 

Jo 


Recall that for low temperatures the chemical potential was claimed to be very 
nearly equal to the Fermi energy. We will use the approximation that p, k, cp now, 
and justify it later on the basis of the results of the Sommerfeld expansion. 
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At low temperatures, /3p « fSep = Tp/T 3> 1. Since the upper limit in the second 
integral in eq. (28.33) is large and the integrand goes to zero exponentially rapidly as 
z —f oo, replacing the upper limit by infinity should be a good approximation. 



(28.34) 



Note that while the replacement of the upper limit of (3^ in eq. (28.33) by infinity 
is almost always an excellent approximation, it is not exact. 

For the next step we will assume that the integrand in eq. (28.34) is analytic, so 
that we can expand the function (j){^ + j3~^z) in powers of z. 



(28.35) 


The assumption of analyticity is essential to the Sommerfeld expansion. While the 
density of states Dpp^e) is generally analytic in some region around the Fermi 
energy, it is not analytic everywhere. This means that there are always small 
corrections to the Sommerfeld expansion, in addition to the one that comes from 
the extension of the upper limit of the integral in eq. (28.33) to infinity. These 
corrections are so small that they are often not even mentioned in textbooks. 
However, if the density of states is zero at e = /i, all terms in the Sommerfeld 
expansion vanish, and only the non-analytic corrections are left. 

Inserting eq. (28.35) in eq. (28.34), we find 




{-zY] {e^ + l)-Uz 




,2n+l 


{e^ + l)~^dz (28.36) 


The integrals on the first line of eq. (28.36) vanish for even values of j, which led 
us to define n = {j — 1)/2 for odd values of j and rewrite the sum as shown in the 
second line. 
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The dimensionless integrals in eq. (28.36) can be evaluated exactly (with a little 
effort) to obtain the first few terms of the Sommerfeld expansion. 

1= tjr{e)de + ^^(i)(^)(A:bT) 2 + g!</,(3)(^)(fc^r)4 + • • • (28.37) 

In eq. (28.37), denotes the first derivative of ((>(e), evaluated at /i, and 

denotes the corresponding third derivative. 

As we will see in the next section, the Sommerfeld expansion converges rapidly for 
low temperatures. 

28.10 General Fermi Gas at Low Temperatures 

As discussed in Section 26.16 of the previous chapter, the first step in calculating the 
properties of a Fermi gas is to use eq. (28.12) to find N a,s a, function of T and ft, 
and then invert the equation to obtain /i = ^(T, N). Using the Sommerfeld expansion, 
eq. (28.37), derived in the previous section, we can expand eq. (28.12) to find iV as a 
power series in T. 

pOO 

N= (exp[/3(e— //)] +1)”^ de (28.38) 

^0 

= £DFD{e)de + (Ai)(fcBT)2 + + ■■■ 

Recall from eq. (28.14) that the Fermi energy is defined as the zero-temperature 
limit of the chemical potential 


N = 


DpEi{d)de 


(28.39) 


Since pt — ep is small at low temperatures, we can approximate the integral in 
eq. (28.38) as 


ppL pep 

/ Dpp{e)defv / Dpp){e)dt +{p — ep)Dpp){ep) 
Jo Jo 


(28.40) 


Putting this approximation into eq. (28.38), we find an equation for the leading term 
in /r — ep. 


2 

M - CF = ~{kBTfD’'^l,ifx)/DpDiep) + ■■■ (28.41) 

We see that the deviation of the chemical potential from the Fermi energy goes to 
zero as for low temperatures, justifying our earlier assertion that is a good 
approximation for /i at low temperatures. 
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The next step is to use eq. (28.41) to find a low-temperature expansion for the 
energy. Defining the zero-temperature energy of the system to be 


Uo 



DFD{€)€de 


(28.42) 


the Sommerfeld expansion of eq. (28.13) can be written as 


U — Uo + — ep)epDFD{.iF) 


^2 r 


6 L 


Inserting eq. (28.41) for fi — ep, this becomes 


Dpoi^p) + ^pOppi^F) {ksT)'^ + 


(28.43) 


-,( 1 ) 


U=Uo- —{kETYepD^iep) 




DfdYf) + ^fD^PdYf) {ksTY + 


(28.44) 


= ^^0 + -^DFD{f-F){kBTY + ■ ■ • 

The heat capacity at constant volume is found by differentiating of eq. (28.44). 


= —DFDYF)kpT + 


\dT 


(28.45) 


N,V 


Eq. (28.45) has two very interesting properties. First, it shows that as long as 
DfdYf) does not vanish, the low-temperature heat capacity is proportional to the 
temperature T. Since the low-temperature contributions of the phonons to the heat 
capacity have a temperature-dependence, we can clearly separate the contributions 
of the phonons and electrons experimentally. 

Next, we have answered the question of why the heat capacity of materials does 
not have a contribution of 3fcB/2 from every particle, whether electrons or nuclei, 
which would be expected from classical theory. We will see this explicitly below in 
eq. (28.53). 

Finally, eq. (28.45) shows that the only thing we need to know to calculate the low- 
temperature heat capacity is the density of states at the Fermi energy. Or, turning the 
equation around, by measuring the low-temperature heat capacity we can determine 
the density of states at the Fermi energy from experiment. 


28.11 Ideal Fermi Gas at Low Temperatures 

While the equations for the specific heat derived so far in this chapter apply to a 
system of fermions with any density of states, it is particularly instructive to look at 
the special case of an ideal Fermi gas. 
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Recall that the density of states for an ideal gas of electrons was given in eq. (28.15). 

3/2 




In eq. (28.46) we defined a constant 


27r2 ) 


?,/2 


to simplify further calculations. 
Eq. (28.38) for N becomes 






= + ^-^A^-kl\ksT)‘ + ... 


- i 


1 + ^ 


2 ^kuTV Ttt^ fkuT^ ^ 


Since 


Ttt" 

6^ 


N=heT 

3 ^ 


Eq. (28.48) gives an equation for ep in terms of /i 


3/2 


3/2 


^2 /ksTV Ttt^ fkBTV 
^ 8 \ pi J ^ 640 \ pi J 


or pi in terms of ep'- 


UK. ep 


ep 


1 + 


7r2 

(keT'^ 

2 

Ttt^ 

8 

K F j 


640 

7r2 

(keT^ 

2 

= ep 

12 

K CF y 



kpT 

M 


^ 12 [TpJ 


- 2/3 


T V 


(28.46) 


(28.47) 


(28.48) 


(28.49) 


(28.50) 


(28.51) 


Since a typical value of Tp is of the order of 6 x lO^^RT, the T2-term in eq. (28.51) 
is of the order of 10“^ at room temperature, confirming that the chemical potential 
is very close to the Fermi energy at low temperatures. This also justifies the omission 
of the next term in the expansion, which is of order (T/Tp)^ k 10“®. 

The heat capacity can be obtained by inserting eq. (28.46) into eq. (28.45). 


Cv = -^rNkp — 


T \ 


TfJ 


(28.52) 



















348 Fermi-Dirac Statistics 


This equation can be rewritten in a form that shows the effect of Fermi statistics on 
the heat capacity. 


The factor in parentheses is the quantum correction, which shows explicitly that the 
Fermi heat capacity is much smaller than that of a classical ideal gas, as claimed in 
the previous section. 

We have seen in Chapter 25 that the specific heat due to lattice vibrations has a 
T^-behavior at low temperatures. Eq. (28.53) shows that when the density of states, 
D{e), is non-zero, there is an additional T-dependence, which dominates at very low 
temperatures. This is indeed observed experimentally for metals. However, it is not 
observed for insulators or semiconductors, for reasons we will discuss in the next 
chapter. 


28.12 Problems 

Problem 28.1 

Fermions in a two-level system (with degeneracy) 

Consider a system of N independent fermions. 

Assume that the single-particle Hamiltonians have only two energy levels, with energies 

0 and e. However, the two levels have degeneracies uq and ni, which are, of course, 

both integers. 

1. First take the simple case of no = ni = 1, with = 1. Find the chemical poten¬ 
tial, /i, as a function of temperature. What is the Fermi energy, ep = = 0)"^ 

2. Now make it more interesting by taking arbitrary values of no and ni, but 
specifying that N = ng. Again find the chemical potential, /r, as a function of 
temperature for low temperatures. That is, assume that /3e 2> 1. What is the 
Fermi energy? 

3. Keep arbitrary values of no and ni, but consider the case oi N < rig. Again find 
the chemical potential, p, as a function of temperature for low temperatures. That 
is, assume that /3e ^ 1. What is the Fermi energy? 

4. Keep arbitrary values of rig and ni, but consider the case oi N > rig. Again find 
the chemical potential, p, as a function of temperature for low temperatures. That 
is, assume that /3e ^ 1. What is the Fermi energy? 

Problem 28.2 

Fermions in a t/iree-level system (with degeneracy) 

Consider a system of N independent fermions. 


Problems 349 


Assume that the single-particle Hamiltonians have three energy levels, with ener¬ 
gies ei, 62 , and £ 3 , where ei < £2 < 63 . The three energy levels have degeneracies rii, 

712, and 713 , which are, of course, integers. The values of the rijS are to be left arbitrary. 

1. First take the case of N = ni. Find the chemical potential, for low tempera¬ 
tures. What is the Fermi energy, ep = fi{T = 0)? 

2. Now take the case of N = ni+n 2 - Find the chemical potential, jj,, for low 
temperatures. What is the Fermi energy, ep = fj,{T = 0)? 

3. Next, consider the situation with 712 > 2 and iV = 711 -|- 1. Find the chemical 
potential, fi, for low temperatures. What is the Fermi energy, ep = ^{T = 0)? 

Problem 28.3 

Ideal Fermi gas in two dimensions 

Consider an ideal Fermi gas in two dimensions. It is contained in an area of dimensions 

L X L. The particle mass is m. 

1. Calculate the density of states. 

2. Using your result for the density of states, calculate the number of particles as 
a function of the chemical potential at zero temperature. (/r(T = 0) = £f, the 
Fermi energy.) 

3. Calculate the Fermi energy as a function of the number of particles. 

4. Again using your result for the density of states, calculate the total energy of the 
system at zero temperature as a function of the Fermi energy, ep . 

5. Calculate the energy per particle as a function of the Fermi energy ep. 

Problem 28.4 

More fun with the Ideal Fermi gas in two dimensions 

Consider the same two-dimensional, ideal Fermi gas that you dealt with in the previous 

assignment. You will need the result of that assignment to do this one. 

1. Calculate the average number of particles as a function of /r exactly. (This is one 
of the few problems for which this can be done.) 

2. Calculate p, as a function of N and T. Then find the high- and low-temperature 
behavior of fi. 

Problem 28.5 

An artificial model of a density of states with a gap in the energy spectrum 

Consider a system with the following density of states 



0 > £ 

£1 > £ > 0 
£2 > £ > £1 
£ > £2 
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where A is a constant, and £2 > ei > 0. 

1. Find the Fermi energy ep = ii{T = 0) for the following three values of the total 
number of particles, N. 


(a) N = AellA 

(b) N = Aell2 

(c) N = SAel/4, 


2. For each of the three cases, find the specific heat at low temperatures from the 
Sommerfeld expansion. 

3. For one of the cases studied above, the Sommerfeld expansion for the spe¬ 
cific heat can be summed exactly to all orders. And the answer is wrong! 
Explain which case is being referred to, and why the Sommerfeld expansion has 
failed. 

Problem 28.6 

An artificial model of a density of states with a gap in the energy 
spectrum—continued 

Again consider a system with the following density of states: 


0 


0 > e 
ei > e > 0 
£2 > e > ei 
£ > £2 



^A(£- £2) 


where A is a constant, and £2 > £1 > 0. The number of particles is given by A = Ae\l2, 
so the Fermi energy is ep = (ci -I- £2)/2. 

In the last assignment we found that the Sommerfeld expansion gives incorrect 
results for this model. In this assignment we will calculate the low-temperature 
behavior correctly. 

1. Show that if t\ ^ kpT, that ft = ep is a, very good approxmation, even if ksT « 
£2 — £i- 

2. Calculate the energy of this model as a function of temperature for low temper¬ 
atures. Assume that £1 ^ £2 — £1 kpT. 

3. Calculate the heat capacity of this model as a function of temperature for low 
temperatures from your answer to the previous question. Assume that £1 ^ £2 — 
£1 > kpT. 



29 

Insulators and Semiconductors 


Mankind is a catalyzing enzyme for the transition from a carbon-based intel¬ 
ligence to a silicon-based intelligence. 

Girard Bricogne 


Although the ideal gas of fermions is a useful model for the behavior of metals, real 
systems deviate from this model in a very important way, in that real energy spectra 
contain gaps in which no energy levels occur. When the Fermi energy lies in one of 
those gaps, the behavior of the electrons is qualitatively different from that described 
in the previous chapter. Such materials do not have the high electrical conductivity 
of metals. They are insulators or semiconductors, and their properties are the subject 
of the current chapter. 

The origin of the strange band gaps that arise in the electron density of states 
in real materials is that the regular arrangement of the nuclei in a crystal lattice 
creates a periodic potential. This affects the energy levels of the electrons and gives 
rise to gaps in the energy spectrum. The mechanism by which the gaps arise can 
be understood from two perspectives. The first argument initially assumes that the 
atoms are very far apart, so that the electrons cannot easily jump from one atom to 
the next. Since an electron is tightly bound to a nucleus, this is known as the ‘tight 
binding approximation’. The second argument for the existence of energy gaps starts 
from the opposite extreme, where the electrons act almost like an ideal gas, but are 
presumed to be subject to a very weak periodic potential. This is known as the ‘nearly 
free electron approximation’. In both cases, gaps in the energy spectrum occur due to 
the periodicity of the lattice. 

In the following sections we will explore both ways of seeing how gaps arise in the 
energy spectrum in a crystal. After that we will investigate the consequences of energy 
gaps, which will lead us to the properties of insulators and semiconductors. 

29.1 Tight-Binding Approximation 

Begin by considering a single, isolated atom. Since we know that electrons in isolated 
atoms have discrete energy levels, the energy spectrum of a single atom will look 
qualitatively like the diagram on the far left of Fig. 29.1. 
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Fig. 29.1 Schematic plot of the energy levels in a crystal. The diagram on the far left shows 
the energy spectrum of an isolated atom. The second diagram shows the effect of bringing two 
atoms close together, and the third diagram corresponds to three neighboring atoms. The diagram 
on the far right shows the band structure that arises when the lattice constant is of the order of 
nanometers, as in a real crystal. The 10^'^ or more levels make each band a quasi-continuum. 

If we bring a second atom to within a few tenths of a nanometer of the first, the 
wave functions will overlap, and the energy levels will be split according to quantum 
perturbation theory, as shown in the second diagram in Fig. 29.1. 

If there are three atoms close together, each energy level will split into three levels, 
as shown in the third diagram in Fig. 29.1. 

Now assume that we have a crystal; that is, a regular array of atoms in which 
the neighboring atoms are separated by a few tenths of a nanometer. If the crystal 
contains 10^° atoms, the energy levels will be split into 10^° closely spaced energy 
levels, which will form a quasi-continuum. Such a quasi-continuum of energy levels is 
called a ‘band’. This is shown schematically in the diagram on the far right of Fig. 29.1. 

In the schematic diagram in Fig. 29.1, the spreading of the energy levels has been 
large enough to merge the top two single-atom energy levels into a single band. 
However, the gap between the bottom two two single-atom energy levels remains, 
despite the spreading of the energy levels into bands. These gaps, or ‘band gaps’, 
represent forbidden energies, since there are no single-particle wave functions with 
energies in the gaps. 

This qualitative description of how gaps arise in the single-particle excitation 
spectrum can be made quantitative by applying perturbation theory to the weak 
overlap of the wave functions of nearly isolated atoms. Since the electrons are tightly 
bound to individual atoms when the overlap is very weak, this is known as the ‘tight- 
binding approximation’. We will not go into the details of the theory here, since our 
purpose is only to understand qualitatively how gaps in the excitation spectrum arise. 

Before going on to discuss the opposite extreme, in which electrons experience 
only a weak periodic potential due to the crystal lattice, we need some mathematics. 
The following section introduces Bloch’s theorem for a particle in a periodic potential 
(Felix Bloch (1905-1983), Swiss physicist, Nobel Prize 1952). Once armed with this 
theorem, we will discuss the effects of the crystal lattice on the single-electron energy 
spectrum. 
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29.2 Bloch's Theorem 

For simplicity, we will again only treat the one-dimensional case. The three- 
dimensional case is not really more difficult, but the necessary vector notation makes 
it look more complicated than it is. 

Consider a single particle in a periodic potential, V{x), so that the Hamiltonian is 
given by 




(29,1) 


Denoting the period by a, we have 

V{x + a) = V{x) (29.2) 

Define translation operators, T„, for any integer n. such that 

TnH{p,x) = H{p,x + na) = H{p,x) (29.3) 

Translation operators have the property that they can be combined. For an arbitrary 
function f{x), 

TnTmfix) = Tnf{x + ma) = f{x + ma + na) = Tm+nf{x) (29.4) 


or 


TnTm = Tm+n (29.5) 

The translation operators also commute with each other 

[Tra, Tm] = TnTm — TmTn = 0 (29.6) 

and every r„ commutes with the Hamiltonian 

[Tn,H]=TnH-HTn = Q (29.7) 

Therefore, we can define wave functions that are simultaneously eigenfunctions of all 
the translation operators and the Hamiltonian. 

Denote a single-particle eigenfunction by (^(x), the eigenvalue of Tn by C^. 

Tn4>{x) = (j}{x -I- a) = Cn4>{x) (29.8) 

From eq. (29.5), we can see that 

TnTm(t){x) = CnCm4>{x) = Tm+nfl^ix) = Cn+m4>{x) (29.9) 


or 


Cn.Cm. — O', 


n-\-m ■ 


(29.10) 
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We can solve eq. (29.10) for Cn in terms of a constant k, 

Cn = exp{ikna) (29.11) 

where A: is a real number. It is easy to confirm that this is a solution by inserting 
eq. (29.11) into eq. (29.10). The reason that k must be real is that (/)(a;) - and therefore 
Cn - must be bounded for very large (positive or negative) values of x. 

The next step is to note that the eigenfunction can be written as 

4>k{x) = exp{ikx)uk{x) (29.12) 

where Uk{x) is periodic. 

Uk{x + a) = Uk{x) (29.13) 

The eigenfunctions (pkix) also have an important periodicity with respect to the 
wave number k. If we define a vector 

O-TT 

K=— (29.14) 

a 

and consider the wave function 

(l)k+K{x) = exp[i(fc + K)x] Uk+K{x) (29.15) 

then 

4’k{x) = exp{ikx)uk{x) (29.16) 

= exp[i(/c + K)x] exp{—iKx\ Uk{x) 

= exp[i(fc + K)x\ Uk+K{x) 

= (l)k+K{.x) 

It is easy to confirm that Uk+K{x) = exp{—iKx)uk{x) is a periodic function of x with 
period a. 

By extending this argument to (j)k+qK{x), where q is any integer, we see that 
must be a periodic function of k with period K = 2tt ja. 

efc = Cfc+ic (29.17) 

The following section uses Bloch’s theorem to calculate the single-particle excita¬ 
tion spectrum for nearly free electrons. The periodicity of Ck will be crucial to the 
discussion. 

29.3 Nearly-Free Electrons 

In this section we will look at the consequences of a weak periodic potential for a gas 
of non-interacting electrons. We will begin by assuming that the potential is exactly 
zero, and then investigate the effects of turning on a periodic potential, such as that 
due to a regular crystal lattice. 
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The discussion will consist of three stages: 

1. Reminder of the energy spectrum of a non-interacting particle in a box. 

2. The effects of Bloch’s theorem on the representation of the energy spectrum of 
non-interacting particles in the limit that the amplitude of the periodic potential 
vanishes. 

3. The effects of turning on a periodic potential with a small amplitude. 


29.3.1 Free Electrons 


We begin with the 
Chapter 26. 

energy levels for free electrons. 

which is given in eq. (26.12) of 


" 2m* 

(29.18) 

Here 

. riTT 

k = — 

L 

(29.19) 


and n is a positive integer. 

Fig. 29.2 shows a plot of Ck as a function of the wave number k. 


29.3.2 The ‘Empty’ Lattice 

The next step is to look at the consequences of the periodicity of the lattice on the 
energy spectrum, while keeping the amplitude of the periodic potential equal to zero. 
This might seem rather strange, since a potential of amplitude zero cannot change the 
energy of a state. Nevertheless, the representation of the energy can change. 

The representation of the energy spectrum in a periodic potential differs from that 
shown in Fig. 29.2 because of the periodicity in fc-space imposed by eq. (29.17). This 
representation is known as the ‘empty lattice,’ because even though we are assuming 
periodicity, there is nothing there to give rise to a non-zero potential. 

The wave functions in the empty lattice representation can be written in the form 
given in eq. (29.12). The empty-lattice energy, ej,, is periodic in fc, as indicated in 
eq. (29.17). The empty-lattice energy spectrum is plotted in Fig. 29.3. 



Fig. 29.2 Free electron energy spectrum. 
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Fig. 29.3 Empty lattice energy spectrum 

Fig. 29.3 shows how the periodicity indicated by eq. (29.17) affects the energy 
spectrum. The parabolic spectrum shown in Fig. 29.2 is repeated at intervals oi K = 
2Tt/a along the entire fc-axis. This duplication has the consequence that all information 
about the energy spectrum is contained in any region of length K = 2Tt/a along the 
axis. It is conventional to choose the region from —itja to tt/o. This region is called the 
Brillouin Zone, after Leon Brillouin (French physicist (1889-1969), American citizen 
after 1949). In Fig. 29.3, the limits of the one-dimensional Brillouin Zone are indicated 
by two vertical dotted lines. 

In more than one dimension, a more general definition is used for the Brillouin Zone. 
The periodicity in A:-space is defined by a set of three vectors, {Ki,K 2 , K^}. The set 
of all vectors of the form riiKi + n 2 K 2 + where the n^s are integers, forms 

a ‘reciprocal lattice’. The Brillouin Zone is then defined as including those points 
closer to the origin in /c-space than to any other point in the reciprocal lattice. For 
the one-dimensional case, the reciprocal lattice consists of the points, 2n7r/a, where 
n is an integer. The points between —tt/o and tt/o then satisfy this definition of 
the Brillouin Zone. 


29.3.3 Non-Zero Periodic Potential 

Now we are going to consider the effects of making the amplitude of the periodic 
potential non-zero. 

If we consider Fig. 29.3, we see that there are crossings of the energy levels where 
k = niTja, where n is an integer. These crossings represent degeneracies, where two 
states have the same energy (in the original representation, just traveling waves going 
in opposite directions). A non-zero periodic potential splits the degeneracy at these 
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Fig. 29.4 Band structure with energy gaps occurring at k = irja, 0, and —nja. 

crossings, according to the usual result of quantum perturbation theory. After the 
splitting, the energy spectrum looks like Fig. 29.4. 

The degeneracies seen in Fig. 29.3 all come from the symmetry that = ^k+K- 
The value of the empty-lattice representation is that it highlights those points at 
which the splitting due to a periodic lattice removes the degeneracies. 


After the splitting of the crossing degeneracies, gaps in the energy spectrum open 
up. There are now ranges of energies for which there are no solutions to the wave- 
function equation. 

29.4 Energy Bands and Energy Gaps 

The consequences of both the tight-binding and the nearly-free electron analyses are 
qualitatively the same. In certain ranges of energies, there exist single-particle energy 
states. These energy ranges are called ‘bands’. In other energy ranges, no such states 
are found. These energy ranges are called ‘gaps’. 

Both the tight-binding and the nearly-free electron analysis can be used as starting 
points for quantitative calculations of the energy spectrum and the density of states. 
However, that would take us beyond the scope of this book. 

The existence of energy gaps is clearly an important feature of the density of states. 
As we will see in the next section, when the Fermi energy falls within a band gap, the 
behavior of the system differs dramatically from what we found for the free-electron 
model. 
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29.5 Where is the Fermi Energy? 

When the energy spectrum has band gaps, the behavior of the material will depend 
strongly on where the Fermi energy is - specifically, on whether the Fermi energy lies 
within a band or within a gap between bands. This, in turn, depends on whether the 
bands near the Fermi energy are empty, completely filled, or partially filled at zero 
temperature. If there is a partially filled band at T = 0, the Fermi energy will lie at 
the boundary between the filled and empty states. If there are only completely filled 
and completely empty bands at T = 0, the Fermi energy will lie halfway between the 
highest filled band and the lowest empty band. 

To understand why the behavior of a material depends strongly on where the Fermi 
energy is, first recall that when a single-particle state is filled, no other electron can 
enter that state. If all single-particle states at a given energy are filled, nothing can 
happen at that energy. Neither can anything happen at an energy for which all states 
are empty. It is only when electrons can change states without changing their energy 
significantly that they can move from one area to another within the system. 

The explicit dependence of the electrical properties of systems on their band 
structures gives rise to the differences between metals, insulators, and semiconductors, 
which will be discussed in the following sections. 

29.6 Fermi Energy in a Band (Metals) 

If the Fermi energy falls within a band, it requires only a tiny amount of thermal 
energy to raise an electron into an empty energy level with a slightly higher energy. 
The electron is then able to move to other states with the same energy in different parts 
of the system. Since electrons are charged, their motion is electric current; systems 
with Fermi energies that fall within a band are metals. 

In Chapter 28 we found from the Sommerfeld expansion that the specific heat at 
low temperatures was given by eq. (28.45). 

Cv = "^DF{tF)klT (29.20) 

For free electrons, this gave the particular result given in eq. (28.52). 

If the Fermi energy lies in one of the energy bands, eq. (29.20) still holds. The only 
change from the free electron result is quantitative, due to the change in the density 
of single-electron states at the Fermi energy. For some important conductors, such 
as copper, silver, and gold, the free electron results are within 40% of the measured 
values of the specific heat at low temperatures. For other metals, the magnitude of the 
change can be quite large—up to an order of magnitude. The details of band structure 
that give rise to these differences can be quite interesting, but go beyond the scope of 
this book. 

If the Fermi energy falls in an energy gap, the behavior of the system is qualitatively 
different from that of a metal, and we find insulators and semiconductors. Their 
properties are dramatically different from those of metals—and from each other! We 
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will consider each case separately in the following sections, but first we will discuss the 
generic equations that govern both insulators and intrinsic semiconductors (defined 
below). 

29.7 Fermi Energy in a Gap 

When there is an energy gap between the highest filled single-particle state and the 
lowest empty one, the Fermi energy must be exactly in the middle of the gap. This is 
a very general principle, which was already implicit in the calculations for the cases 
of discrete sets of energy levels that were treated in Chapter 28. To see that this is so, 
introduce a specific model for which the density of states is given by 


A{e-ecr 


e > ec 
ec > e> ev 
ev > e, 


D{e) 


(29.21) 


0 

B{ev-e)^ 


where A, B, a, and b are positive constants. The usual values of the exponents are 
0 = 6=1 /2—the demonstration of which is left as an exercise for the reader—but for 
now we will keep the notation general. 

The subscripts C and V in (29.21) refer to the ‘conduction’ (upper) and ‘valence’ 
(lower) bands. Conduction will occur only in so far as electrons are thermally excited 
from the valence band into the conduction band. 

Since we are interested in very low temperatures when calculating the Fermi energy, 
states with energies much greater than ec are nearly empty, while those with energies 
much less than ey are nearly full. Therefore, the form of the density of states away 
from the gap is often unimportant. 

It might seem extremely improbable that the filled single-particle state with the 
highest energy (at T = 0) is exactly at the top of a band, while the lowest empty 
state is at the bottom of the next higher band. Actually, it is quite common. The 
reason is most easily seen from the tight-binding picture. A band is formed from 
the splitting of atomic energy levels. If these energy levels are filled, then the bands 
states will also be filled. If the energies in a band do not overlap the energies of any 
other band, it will be completely filled, while the higher bands will be empty. 

As usual, we will use eq. (26.61), which gives the number of particles N as a, 
function of p,, to find the chemical potential. 



(29.22) 


By our assumptions, the lower band is exactly filled at zero temperature. 



(29.23) 
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Subtracting eq.(29.23) from eq.(29.22), we can eliminate N from the equation for 
the chemical potential. 


poo pev p 

0=/ i:)(e)(exp[/3(e-p)] + l)"^de+ / D{e) (exp[/3(e - ^)] + 1)"^ - 1 

en J 0 ' 


de 
(29.24) 


Using the identity from Section 28.3, we can rewrite eq. (29.24) in a form that 
shows the equality between the number of particles taken out of the lower band and 
the number of particles added to the upper band. 



D{e) (exp[/3(e - 


H)] + 1) ^ de 



D{e) (exp[-/3(e - 


m )] + 1 ) ^de 


(29.25) 


Anticipating that /i « e^r = (e^r + eL)/2 at low temperatures, we see that the 
exponentials in the denominators of both integrands are large. This allows us to 
aproximate eq. (29.25) as 


poo pey 

A / (e — ec)“ exp[—/3(e — ^)]de = S / (ey — e)^exp[/3(e —/r)]c?e (29.26) 

J ec ^ —oo 

where we have inserted the approximate form for the density of states in eq. (29.21). We 
have also changed the lower limit of the integral on the right to minus infinity, since the 
exponential in the integrand makes the lower limit unimportant at low temperatures. 

We do not need to evaluate the integrals in eq. (29.26) explicitly at this point, but 
we do need to extract the /3-dependence. We do this by defining new variables for the 
integral on the left of eq. (29.26), 


x = P{e-ec) (29.27) 

and for the integral on the right, 

y=-P{e-ev) (29.28) 


Eq. (29.26) then becomes 

poo 

Aexp{f3{fi — ec)) / x°‘exp[—x\dx = 

Jo 


B 


pOO 

exp(/3(ey - ^))/3"^"M exp[-y]dy (29.29) 
^0 


Now bring the exponentials with to the left side of the equation. 

/ 

exp{P{2fj.- ec - ev)) = 


B [ y^ exp[-y]dy'^ 
Jo _ 

poo 

A / exp[—x]dx 
\ Jo / 


{ksT) 


b—a 


(29.30) 
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Denoting the dimensionless constant X by 

p(X) 

B / y^exp[-y]dy 
Jo 


X = 


poo 

A / exp[—x]dx 
Jo 


we find 


M = ^(ec + ey) + - a) ln(fc{,r) + ^InX 


(29.31) 


(29.32) 


Clearly, the Fermi energy is given by 


cf = lim/r = fiec + ey) (29.33) 

T^O 2 

for any values of the constants A, B, a, and b. 

As mentioned at the beginning of this section, the most usual values of the 
exponents are a = b = 1/2. For this case, eq. (29.32) simplifies. 

^(ec + ey) +(29.34) 


The calculation in this section also gives us an explicit estimate for the number 
of electrons in partially filled energy levels that can contribute to an electric current. 
The number of electrons that are thermally excited to the upper band is given by the 
first term on the right of eq. (29.22). 


poo 

Ne= D{e) {exp[P{€ - y,)] + 1)~^ de 

J en 

D{e) exp[—/3(e — y)]de 

pOO 

exp(-/3(ec - ey)/2) / D{e) exp[-P{e - ec)]de 

J ec 

poo 

exp(-/3(ec - ey)/2)A / (e - exp[-/3(e - ec)]de 

J 

poo 

exp(—/?(ec — ey)/2)A / exp[— 

Jo 

poo 

exp(-/3(ec - ev)/‘2)A{kBT)^^'^ / exp[—x] dx 

Jo 

exp{-P{ec - ey)/2) 


(29.35) 


2 
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The factor exp(—/3(ec' — ey)/2) in the last line of eq. (29.35) shows that the number 
of electrons in the upper band will be exponentially small when the band gap Cgap = 
ec — cy is much greater than ksT. 

For an insulator, the band gap, Cgap, between the highest filled (‘valence’) 
and lowest empty (‘conduction’) band is large; that is, greater than roughly 3eV. 
The reason for the choice of this (somewhat arbitrary) number is that a band 
gap of 3eV implies that at room temperature (T = 300 K), Pegap/2 « 58, so that 
exp{—/3egap/2) « 6.5 x 10“^®. This factor in eq. (29.35) is so small that essentially no 
electrons are found in the conduction band of an insulator with a band gap of 3 eV 
or greater. Without conduction electrons, no current can flow, and the material is an 
insulator. 

29.8 Intrinsic Semiconductors 

A rough definition of a pure (or intrinsic) semiconductor is a material with a band 
structure like an insulator, but with a smaller band gap. Examples are given in 
Table 29.1. 

For an intrinsic semiconductor like germanium (Ge), the factor of exp(—/3egap/2) 
in eq. (29.35) is about exp(—/3egop/2) « exp(—13) « 2 x 10“® at room temperature 
(300 iF). While this value is small, there are still a substantial number of electrons 
that are thermally excited. The conductivity of pure germanium is much smaller than 
that of the good insulator, but it is not negligible. 

29.9 Extrinsic Semiconductors 

The main reason why semiconductors are so interesting—and extraordinarily useful 
in modern electronics—is that their electrical properties are very strongly influenced 
by the presence of relatively small concentrations of impurities. Indeed, that property 
provides a better definition of a semiconductor than having a small band gap. 

Since the word ‘impurities’ tends to have negative connotations that would not be 
appropriate in this context, impurities in semiconductors are generally called ‘dopants’, 
and impure semiconductors are called ‘extrinsic’ semiconductors. There are two basic 


Table 29.1 Band gaps of semiconductors. The third eolumn expresses the band gap in units of 
ksT at room temperature. 


Material 

Egap i^y) 

^gapl ^bT 

Ge 

0.67 

26 

Si 

1.11 

43 

GaAs 

1.43 

55 

GaSb 

0.7 

27 

InAs 

0.36 

24 

InN 

0.7 

27 
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classes of dopant: donors and acceptors. The distinction is based on whether the 
particular dopant has more (donor) or fewer (acceptor) electrons than the intrinsic 
semiconductor. 

In the following two subsections, the two kinds of dopant are discussed. To have a 
concrete example, we will use silicon (Si) in both cases as the intrinsic semiconductor 
that is being doped. 

29.9.1 Donor Impurities 

Phosphorus (P) is an example of a donor impurity for silicon {Si). The phosphorus 
ion, P+, goes into the lattice in place of a Si atom, keeping its net positive charge. The 
remaining electron from the P atom goes into the conduction band. A semiconductor 
that contains donor dopants is called an ‘n-type’ semiconductor because the dopants 
provide extra carriers with negative charge. 

Since there is a Coulomb attraction between the net positive charge on the P+ 
ion in the lattice and the donated electron in the conduction band, the combination 
looks very much like a hydrogen atom. The main differences are that the effective 
mass of electrons in the conduction band differs from that of a free electron, and 
there is a screening effect due to the dielectric constant of Si. The combination of 
these two effects results in a bound state with an energy of about —0.044 eP, relative 
to the bottom of the conduction band. This donor state is close to the bottom of 
the conduction band, since the binding energy is much smaller than the band gap 
of 1.12 eP. Since the extra electron went into the conduction band, the donor-state 
energy lies in the band gap, about 0.044 eP below the bottom of the conduction 
band. 

Since the binding energy is only about half of ksT at room temperature for 
the example of P impurities in Si, even without calculations we can see that 
many more electrons will be excited into the conduction band with dopants than 
without. 

29.9.2 Acceptor Impurities 

Boron (P) is an example of an acceptor impurity for Si. The B atom goes into the 
lattice in place of a Si atom and binds an electron from the valence band, creating a P“ 
ion. Since an electron was removed from the valence band, the band is no longer full. 
The state with the missing electron is known as a ‘hole’. For reasons that go beyond 
the scope of this book, the missing electron acts just like a positive mobile charge. 
A semiconductor that contains acceptor dopants is called a ‘p’-type semiconductor, 
because the dopants provide extra carriers with positive charge. 

In analogy to the situation for donors, the mobile positive hole in the valence band 
is attracted to the negative B~ ion fixed in the lattice. Again, this creates a new 
localized state with an energy in the band gap. However, for ‘p’-type semiconductors 
the state is located just above the top of the valence band (about 0.010 el+ above for 
B in Si). The equations for calculation of the energy are essentially the same as those 
for the bound state in ‘n’-type semiconductors. 
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29.10 Semiconductor Statistics 

Although electrons are fermions, so that we might expect the equations for Fermi 
statistics to apply to the impurity states, the truth is rather more interesting. The 
occupation of the donor and acceptor states is determined by what we might call 
‘semiconductor statistics’, which we have not seen before in this book. 

The reason for the difference between semiconductor statistics and Fermi statistics 
arises from two features of the impurity energy levels: 

1. The impurity levels in doped semiconductors are two-fold degenerate because 
there are both spin-up and spin-down states. 

2. The two states in a given level cannot both be occupied, because of a strong 
Coulomb repulsion. 

These features mean that the probabilities for occupying the two degenerate states 
that make up the impurity energy level cannot be regarded as independent: the 
possibility of the energy level containing two electrons is excluded. Because of the lack 
of independence of the two impurity states, the derivation of semiconductor statistics 
given in the next subsection calculates the occupation of the entire energy level. 

29.10.1 Derivation of Semiconductor Statistics 

To derive semiconductor statistics—that is, the occupation number for an impurity 
energy level—we return to eq. (26.31) in Chapter 26. 


Z = ^ exp[-/3(e - 


(29.36) 


For an donor energy level with energy e^, the sum over rie^ must only contain terms 
corresponding to (1) both states empty, (2) only the spin-up state occupied, and (3) 
only the spin-down state occupied. Double occupancy of the donor energy level is 
excluded due to the Coulomb repulsion energy. 


(29.37) 


^exp[-/3(ed - n)n^] = exp[0] -I- exp[-/3(ed - /r)] -I- exp[-/3(ed - /r)] 


1 + 2exp[-P{ed - /r)] 


Inserting eq. (29.37) in eq. (26.44) for the average occupation number, we find 


J2r,,neexp[-f3{e- ft)ne] 
En, exp[-/3(e - fi)ne] 
2exp[-f3{ed - ii)] 


(29.38) 


1 -k 2exp[-(3{ed - /.t)] 
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The extra factor of 1/2 in the denominator of the last line in eq. (29.38) distinguishes 
semiconductor statistics from fermion statistics. Naturally, the single-particle states 
in semiconductor electron bands still obey Fermi-Dirac statistics. 

29.10.2 Simplified Illustration of Semiconductor Statistics 

As an example of the consequences of semiconductor statistics, we will first consider a 
simplified model in which there are N^i donor impurities with a corresponding energy 
level Cd, which lies slightly below the bottom of the conduction band at ec- We 
will assume that the valence band is filled, so that we can ignore it. To simplify 
the mathematics and the interpretation of the results, the conduction band will be 
approximated by assigning all Nc states the same energy, ec —not because it is realistic 
(it is not), but to show the effects of semiconductor statistics. A more realistic model 
will be presented in the next subsection. 

As was the case for problems with either Fermi-Dirac or Bose-Einstein statistics, 
the first task is to find the Fermi energy and the chemical potential. The equation 
governing the chemical potential is, as usual, the equation for the total number of 
particles. Since each donor impurity contributes a single election, the total number of 
electrons is equal to Nd, the number of donor levels. 



Note that the electrons in the conduction band still obey Fermi-Dirac statistics. 

We might have imagined that the Fermi energy coincides with the energy of the 
impurity level, e^, because of the two-fold degeneracy and single occupancy at T = 0. 
However, due to the exclusion of double occupancy of the impurity level, the Fermi 
energy turns out to be half way between the impurity-state energy and the bottom 
of the conduction band, ep = (ec + ed)/2. We will assume that this is true in the 
following derivation of the Fermi energy, and confirm it by the results of the calculation. 

To solve eq. (29.39), we can use a variation on the identity in eq. (28.7). 


iVd [2exp[-/3(ed - m)] + 1] ^ [exp[/3(ec -/^)] + 1] ^ (29.40) 


At low temperatures, since we expect ^ « (cd -I- ec)/2, we have both — e^) ^ 1 
and (3(ec — fj.) ^ 1. This means that the exponentials dominate in both denominators 
in eq. (29.40), and the neglect of the in both denominators is a good approxima¬ 
tion. 


Na 

— exp[/3(ed - m)] ~ Nc exp[-/3(ec - A^)] 


(29.41) 


We can solve this equation for the chemical potential. 



(29.42) 
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Since the Fermi energy is the zero temperature limit of the chemical potential, we 
can immediately see from eq. (29.42) that 




eg + Cd 
2 


(29.43) 


On the other hand, because the difference in energy between ec and can be of the 
order of fesT, the second term in eq. (29.42) is not always negligible. There are usually 
far fewer impurities than conduction states, and we can easily have Nii « 10“^iVc. The 
second term in eq. (29.42) would be roughly —12 kgT. To see how significant this effect 
is, we need to calculate the occupancy of the donor levels. 


{ud) = Nd 


exp [13 {ed - /r)] + 1 


-1 


Inserting eq. (29.42) for the chemical potential, we find 
T 


(ud) « Nd 


exp 


^ , ec + ed ^ 1 f Nd 


1 -1 


or 


(ud) « Nd 


1 /2jVc\ 

2 \ Nd J 


1/2 


exp 


-p 


ec — ed 


+ 1 


+ 1 


(29.44) 


(29.45) 


(29.46) 


There are two competing factors in the first term in the denominator of eq. (29.46). 
While the factor of exp [— /3(ec — ed)/2] can be small for very low temperatures, for 
As in Ge at room temperature it has a value of about 0.6. On the other hand, if 
Nd « 10-5/Vc, the factor of ^/2Nc/Nd « 450, so that it would dominate, making the 
occupation of the donor state quite small. Under such circumstances, almost all of the 
donor levels would be empty, and the electrons would be in the conduction band. 

Note that the emptying of the donor levels is an entropic effect. There are so many 
more states in the conduction band that the probability of an electron being in a 
donor level is very small, even though it is energetically favorable. This feature is not 
an artifact of the simple model we are using for illustration in this section; it is a real 
feature of semiconductor physics. 

In the next section we will consider a (slightly) more realistic model that removes 
the unphysical assumption of an infinitely thin conduction band. 


29.10.3 A More Realistic Model of a Semiconductor 

Although our neglect of the valence band in the model used in the previous subsection 
is often justifiable, giving all states in the conduction band the same energy is certainly 
not. In this subsection, we will introduce a model that is realistic enough to bear 
comparison with experiment - although we will leave such a comparison to a more 
specialized book. 

We will assume the donor levels all have energy ed, as in Subsection 29.10.2. 
However, we will model the conduction band by a density of states as in eq. (29.21). 
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The total density of states can then be written as 


D{e) 


A{e - e > ec 
NdS{e-ed) e < ec 


(29.47) 


Since only single-particle states near the bottom of the conduction band will be 
occupied, the upper edge of the band is taken to be infinity. 

The equation for the total number of donor electrons, which gives us the chemical 
potential, is then 


1 -1 


- exp[/3(ed - Ai)] + 1 


Nd = Nd 

z 

poo 

+ A{e-ecY^'^ (exp[/3(e - A^)]-k 1)"^ - 1 


(29.48) 


de 


As in the previous section, we will again use a variation on the identity in eq. (28.7) 
to simplify the equation. We will also introduce the same dimensionless variable defined 
in eq. (29.27). 


Nd [2 exp[-/3(ed - a ^)] -k 1 ] ^ 


Aexp{P{fi- ec)) (3 ^ “ 



exp[—x\dx (29.49) 


Since we can evaluate the integral, this becomes 


Nd[2exp[-(3{ed-p)] + l] ^ exp {P{n - ec)) (29.50) 

The Fermi energy again turns out to be half way between the energy level of the 
donor state and the bottom of the conduction band. 


ep 


ed + ec 
2 


(29.51) 


The remaining behavior of this more realistic model is qualitatively the same as 
that of the simplified illustration in the previous subsection. The details will be left 
to the reader. 


29.11 Semiconductor Physics 

This brings us to the end of our introduction to insulators and semiconductors. The 
field of semiconductors, in particular, is extremely rich. We have covered the bare 
basics, but we have not even mentioned how semiconductors can be used to create 
lasers or the transistors that are the basis of all modern electronics. However, this 
introduction should be sufficient preparation to read a book on semiconductors without 
too much difficulty. The field is both fascinating and extraordinarily important in 
today’s world. 








30 

Phase Transitions and the Ising 
Model 


Magnet, n. Something acted upon by magnetism. 

Magnetism, n. Something acting upon a magnet. 

The two definitions immediately foregoing are condensed from the works of 
one thousand eminent scientists, who have illuminated the subject with a great 
white light, to the inexpressible advancement of human knowledge. 

Ambrose Bierce, in The Demi’s Dictionary 


The Ising model is deceptively simple. It can be defined in a few words, but it displays 
astonishingly rich behavior. It originated as a model of ferromagnetism in which the 
magnetic moments were localized on lattice sites and had only two allowed values. This 
corresponds, of course, to a spin-one-half model, but since it is tiresome to continually 
write factors of h/2, it is traditional to take the values of a ‘spin’ aj, located on lattice 
site j to be 


cTj = +l or -1 (30.1) 

The energy of interaction with a magnetic field also requires a factor of the magnetic 
moment of the spin, but to make the notation more compact, everything will be 
combined into a single ‘magnetic field’ variable h, which has units of energy. 

Hh = -haj (30.2) 

Eq. (30.2) implies that the low-energy state is for (jj = -|-1 when h > 0. 

There should not be any confusion of the ‘h’ in eq. (30.2) with Planck’s constant, 
because the latter does not appear in this chapter. 


The energy of interaction between spins on neighboring lattice sites, j and k, is 
given by the product of the spins. 


Hj — jGjCTji 


( 30 . 3 ) 
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The constant J is also taken to have units of energy. The low energy states of 
eq. (30.3) are cr^ = cr^ = +1 and <Tj = cr^. = — 1. This form of interaction is also called 
an ‘exchange’ interaction, due to the role of exchanging electrons in its derivation from 
the quantum properties of neighboring atoms. That derivation is, of course, beyond 
the scope of this book. 

If we put the two kinds of interactions together on a lattice, we can write the 
Hamiltonian as 


N 

H = — J Ujcrk — h cTj (30.4) 

where the notation (j, k) denotes that the sum is over nearest-neighbor pairs of sites. 
In the rest of this chapter we will discuss the remarkable properties of the model 
described by eq. (30.4). 

30.1 The Ising Chain 

The first calculations on the Ising model were carried out by Ernst Ising (German 
physicist, 1900-1998), who was given the problem by his adviser, Wilhelm Lenz 
(German physicist, 1888-1957). In 1924 Ising solved a one-dimensional model with 
N spins, in which each spin interacted with its nearest neighbors and a magnetic field. 

H=-J E E-. (30.5) 

3 3 

This model is also referred to as an ‘Ising chain’. The limits on the first sum have not 
been specified explicitly in eq. (30.5) to allow it to describe two kinds of boundary 
conditions. 

1. Open boundary conditions 

The first sum goes from j = ltoj = N — 1. This corresponds to a linear chain 
for which the first spin only interacts with the second spin, and the spin at j = N 
spin only interacts with the spin at j = iV — 1. 

2. Periodic boundary conditions 

A spin (Tat- 1-1 = is defined, and the first sum goes from j = 1 to j = N. This 
corresponds to a chain of interactions around a closed loop. 

In both cases, the second sum runs from j = 1 to j = iV to include all spins. 

We will investigate various aspects of this model in the following sections before 
going on to discuss the (qualitatively different) behavior in more than one dimension. 
We begin by ignoring the interactions and setting J = 0 in the next section. 


30.2 The Ising Chain in a Magnetic Field (J = 0) 

Begin by setting J = 0 in eq. (30.5). 
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N 




i=i 

The canonical partition function is then given by 

N 

Z = '^exp[-f3{-h'^crj)] 

{<7} i = i 


(30.6) 


(30.7) 


where we have used the notation that {a} = {aj |j = 1, • • • , fV} is the set of all spins. 

Eq. (30.7) can be simplified by using the best trick in statistical mechanics 
(factorization of the partition function: Sections 19.12 and 23.7). 


where 


N N N 

^ = E n = n E 

{cr} j=l j=l aj j=l 


Zj = exp[f3hc 


(30.8) 


(30.9) 


This reduces the iV-spin problem to N identical single-spin problems, each of which 
involves only a sum over two states. 

Since each aj = ±1, the sums can be carried out explicitly. 


Zj = exp(/3h) -I- exp(—/3/i) = 2cosh(/3/i) 

It is easy to show that the average magnetization is given by 

exp(/3h) — exp(—/3h) 


m = m,- = (cr,) = 


exp(/3h) -I- exp(—/3/i) 


= tanh(/3/i) 


(30.10) 


(30.11) 


Since it will be important later to be familiar with the graphical form of eq. (30.11), 
it is shown in Fig. 30.1. The shape of the curve makes sense in that it shows that m = 0 
when h = 0, while m —*■ ±1 when h —>■ ±oo. 

The average energy of a single spin is then easily found. 

Uj = {—hcTj) = —hrrij = —hm = —htanh(/3h) (30.12) 

The energy of the entire Ising chain is given by simply multiplying by N. 

N 

U = (H) = —h''^^{aj) = —hNm = —hN ta,iah{(3h) (30.13) 

i=i 

The specific heat is found by differentiating with respect to the temperature. 


1 dU 


-1 dU 


N dT NkeT^ 5/3 


= fcB/3^/i^sech^(/3/i) 


(30.14) 
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m 



Fig. 30.1 Plot of the magnetization of an isolated spin against the magnetic field, 
m = tanh(/3/i). 

The specific heat goes to zero at both T = 0 and T = oo. 

30.3 The Ising Chain with h = 0, but J 7 ^ 0 

Next, we will put the interactions between spins back into the Ising chain, but remove 
the magnetic field. To simplify the problem we will use open boundary conditions. 


N-l 



(30.15) 


At first sight, finding the properties in this case looks considerably more difficult 
than what we did in the previous section. Each spin is linked to its neighbor, so the 
Hamiltonian cannot be split up in the same way to factorize the partition function. 
However, we can factorize the partition function if we introduce new variables. 

Let Ti = CTi, and Tj = aj-iaj for 2 < j < N. Clearly, each Tj takes on the values 
±1. Given the set of N values {Tj\j = 1,... ,N}, we can uniquely find the original 
values of the aj’s. 

The Hamiltonian in eq. (30.15) can now be rewritten as 


N 



(30.16) 


which makes the Hamiltonian look just like eq. (30.6), which we derived in the previous 
section for the case of independent spins in a magnetic field. Although the — 1 
T-variables all appear in the Hamiltonian, cti does not. Summing over cti = ±1 in the 
partition function is responsible for the overall factor of two in the equations below. 
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Because eq. (30.16) is a sum of independent terms, the partition function can again 
be calculated by factorization. 

N N N 

Z = ^ n exp[/3Jr,] = 2 ^ exp[/3Jr,] = 2 Z'j (30.17) 

{t} i = l j=2 Tj j=2 

where 

Zj = ^ exp[/3 Jtj] = exp[/3J] — exp[—/3Jrj] = 2 cosh(/3J) (30.18) 

Tj=±l 

and 

Z = 2(Zj)^-i = 2 (2cosh(/3J))^”^ (30.19) 

The free energy of the Ising chain with h = 0 is given by the logarithm of the 
partition function in eq. (30.19). 

-PF = In Z = In 2 + (iV - 1) In Zj = In 2 + (IV - 1) In (2 cosh(/3 J)) (30.20) 

The energy of the system is easily found to be 

{7= -(1V_ i)jtanh(/3J) (30.21) 

and the specific heat per spin is 

c = J^sech^(/3J) (1 — 1/iV) « J^sech^(/3J) (30.22) 

Note that the form of eq. (30.22) is essentially the same as that of eq. (30.14), which 
we found in the previous section. The specific heat in this case also goes to zero at 
both T = 0 and T = oo. 

30.4 The Ising Chain with both J ^ 0 and h ^ 0 

The full Hamiltonian in eq. (30.5) with both exchange interactions and a magnetic 
field is more challenging. We cannot factorize the partition function in the same way 
that we have done with other problems, so we will have to use a different approach, 
known as the Transfer Matrix method. The basis of the method is that the partition 
function can be expressed as a product of relatively small matrices—in this case, just 
2x2 matrices—which can then be diagonalized and their determinants evaluated. 

30.4.1 Transfer Matrix 

We will consider the case of periodic boundary conditions, so that an+i = cri, as 
discussed earlier. To make the matrices more symmetric, we will write the Hamiltonian, 
eq. (30.5), in the form 

N ^ N 

H=-JJ2 ^^3 + ^3 + 1) 

1=1 ^ 1=1 


(30.23) 
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The upper limit on the first summation is N, so that the last term is giving 

us periodic boundary conditions. 

Using eq. (30.23), we can transform the partition function into a product of 2 x 2 
matrices. 


Z = ^ exp ^-/3 J CTj+i) j j 

= ^ (l3JcrjCrj+i + {aj + cTj+i) 


{<^1 i 


In this equation, 


T{aj,aj+i) = exp ^pjajaj+i + (ctj +crj+i)^ 


(30.24) 


(30.25) 


is defined as the transfer matrix. Note that we have not exchanged the sums and 
products, because we have not eliminated the coupling between variables. The sums 
remain and are interpreted as matrix multiplication. As the name implies, we can also 
write the transfer matrix explicitly as a matrix. 


/ exp(/3J + ph) exp(—/3J) \ 

y exp(—/3 J) exp(/3 J — ph) j 


(30.26) 


Eq. (30.24) is equivalent to representing the partition function as a product of 
matrices. To see this more clearly, consider the factors involving only three spins, 
somewhere in the middle of the chain, at locations j — 1, j, and j + 1. 

(30.27) 

<7j=±l 

The expression on the left of eq. (30.27) can also be written in terms of the explicit 
matrices from eq. (30.26). 


/ exp(/3J + ph) exp{—pj) \ / exp{pj + ph) exp(—/3J) \ 

y exp(—/3J) exp(/3J — ph) J y exp(—/3J) exp(/3J — ph) J 


Carrying out the matrix multiplication explicitly, we find 

r^(crj_i, cTj+i) = 


I exp(2pj + 2ph) + exp(—2/3J) exp{ph) + exp{—ph) \ 

y exp{ph) + exp(—/3/i) exp(2/3 J — 2ph) + exp(—2/3 J) j 

(30.29) 
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If we carry out the sums over a 2 through ctjv, we find the matrix T^((Ti, ctat+i). 
Since ctn+i = the final sum over di yields the partition function. 

Z = T^(cri,CTi) (30.30) 

(Tl—±1 


Fortunately, we do not have to carry out all the matrix multiplications explicitly. 
That is the power of the Transfer Matrix method—as we will show in the next section. 

30.4.2 Diagonalizing r(crj, crj_|_i) 

The next step is the thing that makes this calculation relatively easy. The trace is 
invariant under a similarity transformation of the form 

f = RTR-^ (30.31) 

In this equation, R and its inverse R~^ are both 2x2 matrices. If we write out the 
product of T’s we can see how it reduces to the original product of T’s. The series of 
products brings the R and R~^ matrices together, so that their product gives unity. 
Using the compact notation, T{aj, ctj+i) T^j+i, we see that 

ri.2T^,3^3.4 = RTi^2R-^RT2,3R~^RT3,iR-^ = RTi^2T2,3T3,4R~^ (30.32) 

This procedure can be extended to arbitrarily many T matrices. Since we have 
periodic boundary conditions, after applying the transformation eq. (30.31) N times 
and using R~^R = 1 between the T-matrices, eq. (30.30) becomes: 

Z = TrRT^R-^ = Trf^ (30.33) 

For the last equality, we have used the fact that the trace of a product of matrices is 
invariant under a cyclic permutation. 

Finding the trace is much easier if we capitalize on the freedom of making a 
similarity transformation to diagonalize the T matrix. The diagonal elements of T' 
will then be given by the eigenvalues A+ and A_. 

r(a„a,+i)= (30.34) 

The product of N diagonal matrices is trivial 

I^A^ ^0^^ (30.35) 

and the partition function becomes 


Z = \^ + X’l 


(30.36) 
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The only remaining task is to find the eigenvalues of the T matrix in eq. (30.25). 
This involves solving the eigenvalue equation 


Tx = Ax 


(30.37) 


or 


exp(/3J + j3h) exp(—/3J) \ 
exp(—/3J) exp(/3J — /J/i) / \ X 2 


= A 


(30.38) 


This matrix equation is equivalent to two simultaneous equations for A, and can be 
solved by setting the determinant equal to zero. 


exp(/3J + /3/i) — A exp(—/3J) 
exp(—/3 J) exp(/3J — (3h) — A 


= 0 


(30.39) 


Solving the resulting quadratic equation—and leaving the algebra to the reader— 
we find the required eigenvalues. 

A± = cosh(/3/i) ± y^e^^"^cosh^(/3/i) — 2sinh(2/3J) (30.40) 

The partition function of a system of N spins is given by inserting eq. (30.40) into 
eq. (30.36). 

It is interesting to see how this form of the partition function behaves in the limit 
that N ^ 00 (the thermodynamic limit). Since A+ > A_, the ratio A_/A+ < 1. We 
can use this fact to rewrite eq. (30.36) in a convenient form. 


Z=X^ 






(30.41) 


The free energy then takes on a simple form. 

F = -fcsTlnZ 


= —ksTNln — ksTIn | 1 + ( 


N\ 


—ksTNln X+ — ksT ( — 


N 


(30.42) 


The free energy per spin in the thermodynamic limit is just 

N 


— « -kBTlnX+ - — ( — 
N ^ N VA+ 


—fc^Tln A+ 
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or, inserting A+ from eq. (30.40), 

lim ^ = —fc^Tln ( cosh(/3/i) + -J cosh^{(}h) — 2sinh(2/3J) ) (30.43) 

N—>-00 N \ ^ J 

Perhaps the most important thing to notice about eq. (30.43) is that the free energy 
is an analytic function of the temperature everywhere except at T = 0. Since a phase 
transition can be defined as a point at which the free energy is not analytic in the 
limit of an infinite system (thermodynamic limit), this means that the one-dimensional 
Ising model does not exhibit a phase transition at any non-zero temperature. 

This lack of a phase transition was one of the main results that Ising found (by a 
method different from that which we have used). Based on this result, he speculated 
that the model would not show a phase transition in higher dimensions, either. In 
this speculation, he was wrong. However, it took from 1924, when Ising did his thesis 
work, until 1944, when Lars Onsager (Norwegian chemist and physicist, 1903-1976, 
who became an American citizen in 1945) derived an exact solution for the two- 
dimensional Ising model (without a magnetic field) that showed a phase transition. 

Although Onsager’s exact solution of the Ising model is beyond the scope of this 
book, we will discuss some approximate methods for calculating the behavior of the 
Ising model at its phase transition, beginning with the Mean Field Approximation in 
the next section. 

30.5 Mean Field Approximation 

The history of the Mean Field Approximation (MFA) is interesting in that it precedes 
the Ising model by seventeen years. In 1907, Pierre-Ernest Weiss (French physicist, 
1865-1940) formulated the idea of describing ferromagnetism with a model of small 
magnetic moments arranged on a crystal lattice. He assumed that each magnetic 
moment should be aligned by an average ‘molecular field’ due to the other magnetic 
moments, so that the idea was originally known as the ‘Molecular Field Theory’. The 
justification of the model was rather vague, but comparison with experiment showed 
it to be reasonably correct. 

In the following subsections we will derive the Mean Field Approximation for the 
Ising model, as given in eq. (30.4), in an arbitrary number of dimensions. 

N 

H = — J CTjCrfc — Gj (30.44) 

O'A) f=i 

Recall that the notation (j, k) indicates that the summation in the first term is taken 
over the set of all nearest-neighbor pairs of sites in the lattice. 

30.5.1 MFA Hamiltonian 

The basic idea of MFA is to approximate the effects of interactions on a spin by 
its neighbors as an average (or ‘mean’) magnetic field. Specifically, we would like to 
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calculate an approximation to the average magnetization of a spin at site j. 


rrij = {aj) 


(30.45) 


For simplicity, we will assume periodic boundary conditions, so that the properties 
of the model are translationally invariant. Since all spins see the same average 
environment, they should all have the same average magnetization, m. 


m = 



(30.46) 


For any given spin, dj, we can express the sum over the nearest-neighbor sites as 
a sum over the neighbors 6, and extract all terms in eq. (30.44) that contain aj. 


Hj — haj — 


j+S 


(30.47) 


Although eq. (30.47) can be regarded as a reduced Hamiltonian for spin aj, we 
cannot interpret its average value as representing the average energy per spin, since 
H 7 ^ Hj because of double counting the exchange interactions. This is a mental 
hazard that has caught many students and even a few textbook authors. 


Note that eq. (30.47) has the same form as the Hamiltonian for a single spin in a 
magnetic field, —ha, from eq. (30.2), except that the external magnetic field is replaced 
by an effective magnetic field 


^3,eff — J ^ , '^j+s 4" ^ (30.48) 

<5 

The difficulty with the effective field found in eq. (30.48) is that it depends on the 
instantaneous values of the neighboring spins, which are subject to fluctuations. MFA 
consists of replacing the fluctuating spins in eq. (30.48) by their average values, which 
must all be equal to the magnetization m, due to translational invariance. 

= J y^ {ajj.s) + h = Jzm + h (30.49) 

s 

In the second equality in eq. (30.49) we have introduced the parameter z to denote the 
number of nearest-neighbor sites in the lattice of interest (z = 4 for a two-dimensional 
square lattice). The MFA Hamiltonian, 

jjMFA ^ _^MFA^^ (3Q 
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has exactly the same form as eq. (30.2). An equation for the magnetization can be 
written down immediately in analogy to eq. (30.11). 

m = tanh(/3h^'^"^) = ta,nh{P Jzm + Ph) (30.51) 

Unfortunately, there is no closed-form solution for eq. (30.51). However, there is 
a nice graphical solution that makes it relatively easy to understand the essential 
behavior of the solution, as well as guiding us to good approximations. 

30.5.2 Graphical Solution of eq. (30.51) for h — 0 

First consider the case of a vanishing magnetic field, h = 0, for which eq. (30.51) 
simplifies. 


m = ta,nh{P Jzm) (30.52) 

It might seem strange, but we can further simplify the analysis of eq. (30.52) by 
expressing it as two equations. 


m = tanh(a;) (30.53) 

X = pjzm (30.54) 

The second equation can then be rewritten as 

(30.55) 

Since eqs. (30.53) and (30.55) are both equations for m as a function of x, it is 
natural to plot them on the same graph, as shown in Fig. 30.2. Intersections of the 
two functions then represent solutions of eq. (30.52). 

When the slope of the line described by eq. (30.55) equals 1, it provides a dividing 
line between two qualitatively different kinds of behavior. The temperature at which 
this happens is known as the critical temperature, Tc, or the Curie temperature, in 



m 



Fig. 30.2 Plot of m = tanhx and m = {T/Tf)x for T > Tc- 
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m 



Fig. 30.3 Plot of m = tanh® and m = {Tc/T)x for T <Tc. 


honor of Pierre Curie (French physicist, 1859-1906, awarded the 1903 Nobel Prize, 
together with his wife, Maria Sklodowska-Curie, 1867-1934). 


(30.56) 


fcsTc = zJ 


This expression for the mean field value of Tc means that eq. (30.55) can be rewritten 
as 



(30.57) 


If T > Tc, the slope of the straight line described by eq. (30.57) is greater than 1, 
and the two curves in Fig. 30.2 will only intersect at m = 0 and a; = 0. This gives the 
reasonable result that the magnetization above Tc is zero. 

For temperatures below Tc there are three intersections of the two curves, as shown 
in Fig. 30.3. The intersection at m = 0 is not stable, which will be left to the reader 
to show. The other two intersections provide symmetric solutions m = ±to(T), which 
are both stable. 

The two solutions found in Fig. 30.3 are expected by the symmetry of the Ising 
Hamiltonian under the transformation aj —>■ —aj for all spins. 

30.5.3 Graphical Solution of eq. (30.51) for h ^ 0 

The situation is qualitatively different when the magnetic field, h, is non-zero. The 
Ising Hamiltonian, eq. (30.4), is no longer invariant under the transformation aj —>■ 
—aj for all spins, because the second term changes sign. Eq. (30.51) can again be 
simplified by writing it in terms of two equations. 


m = tanh(a;) 

X = (3Jzm + f3h 


(30.58) 

(30.59) 
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m 



Fig. 30.4 Plot of m vs. x and m = {Tc/T)x — h/ksT for T < Tc. 


The second equation can then be rewritten as 


m = 



h 


(30.60) 


Fig. 30.4 shows a plot of eqs. (30.58) and (30.59) on the same graph. In analogy 
to the previous section, the intersections of the two functions represent solutions to 
eq. (30.11). 

In Fig. 30.4 , T < Tc and h > 0. Of the three points of intersection that mark 
solutions to eq. (30.11), only the one at the far right, with m > 0, is stable. The 
intersection to the far left is metastable, and the intersection nearest the origin is 
unstable. (Proofs will be left to the reader.) 

It can be seen by experimenting with various parameters that there is always a 
stable solution with m > 0 when h > 0. Similarly, when h < 0 there is always a stable 
solution with m < 0. 

As long as h 0, the MFA prediction for the magnetization is an analytic function 
of the temperature, and there is no phase transition. This property is also true for the 
exact solution; a phase transition appears only in the absence of an applied magnetic 


field. 


30.6 Critical Exponents 

A curious aspect of phase transitions is that many properties turn out to have some sort 
of power-law behavior. For example, the magnetization near the critical temperature 
can be approximated by the expression 


m!=sA\T-Tcf 


(30.61) 


where A is a constant. 

There is a serious source of potential confusion in eq. (30.61). The use of the symbol 
/3 as a dimensionless critical exponent is in direct conflict with the use of the 
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same symbol as j3 = l/ksT. Unfortunately, both uses of (3 are so firmly rooted 
in tradition that an alternative notation is impossible. I can only council patience 
and a heightened degree of alertness when you encounter the symbol /3 anywhere 
in statistical mechanics or thermodynamics. 


There is a full Greek alphabet soup of critical exponents defined in a similar 
manner. For example, for the specific heat. 


c (X |T — Tc 

and for the magnetic susceptibility y. 



/ dm 

X = 



/ dm 

X = 



1-7 


h=0 


T=Tc 


,1/S 


(30.62) 


(30.63) 

(30.64) 


Correlations between spins can be described by a correlation function, which 
gives rise to additional critical exponents. The correlation function has the following 
asymptotic form for large separation r = |r) — between spins. 

/(r) = (cTjO-fc) - {(Tj){ak) oc exp[-r/^] (30.65) 


The correlation length which is the characteristic size of fluctuations near r^, 
diverges with an exponent v, 


^(x\T-T^\~^ (30.66) 

and an exponent rj that describes the power-law fall-off of the correlation function in 
eq. (30.65). 

All of these critical exponents can be evaluated in MFA, as illustrated in the 
following subsection. 


30.7 Mean-Field Exponents 

We can derive the value of /3 within the mean field approximation by making use of 
the expansion of the hyperbolic tangent in eq. (30.67). 


To 

tanhx = X — -x 
o 


(30.67) 


Near the critical point, the magnetization m is small, so that the argument of the 
hyperbolic tangent in eq. (30.52) is also small. We can therefore use eq. (30.67) to 
approximate eq. (30.52). 


771 = tanh(/3Jzr?7) « pjzm 


H- 


(30.68) 
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One solution of eq. (30.68) is obviously m = 0, which is the only solution above Tc- 
Below Tc, the non-zero solutions for the magnetization are stable, and to find them we 
can divide eq. (30.68) by m 0. Solving the resulting equation for the magnetization, 
we find 



Close to Tc, the magnetization behaves as 



so that we find 


Pmfa = 2 


(30.69) 


(30.70) 


(30.71) 


The calculation of each of the critical exponents can be carried out within the 
mean field approximation. However, it is only fair to let the reader have some 
of the fun, so we will leave the evaluation of the other critical exponents as 
exercises. 


30.8 Analogy with the van der Waals Approximation 

In Chapter 17 we discussed the van der Waals fluid and showed that it predicted a 
transition between liquid and gas phases. Although we did not emphasize it there, 
the van der Waals fluid is also an example of a mean field theory. The interactions 
between particles are approximated by an average (mean) attraction strength (rep¬ 
resented by the parameter a) and an average repulsion length (represented by the 
parameter b). 

In the van der Waals approximation, the difference in density between the liquid 
and gas phases was found to be given by the expression 

PH<, - Pgas Ot \T - (30.72) 


close to the critical point. 

Eq. (30.72) for the discontinuity in density in the vdW fluid is directly analogous 
to eq. (30.61) for the discontinuity in magnetization in the Ising model from +\m\ to 
— \m\ as the applied magnetic field changes sign. Indeed, the analogy between these 
two models can be extended to all critical properties. Just as the density discontinuity 
is analogous to the magnetization, the compressibility is analogous to the magnetic 
susceptibility. 

The analogy between the vdW fluid and the MFA Ising model even extends to both 
models having the same value of the critical exponent with a value of jSydw = 1/2 = 
Pm FA- Indeed, all critical exponents, including those to be calculated as homework, 
are the same in both models, which will be discussed in the next section. 
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30.9 Landau Theory 

The great Russian physicist, Lev Davidovich Landau (1908-1968) made a remarkable 
generalization of the mean field approximation. His theory included not only the van 
der Waals and Ising models that we have discussed, but any extension of them with 
more distant or more complicated interactions. 

Landau first introduced the concept of an ‘order parameter’; that is, an observable 
quantity that describes a particular phase transition. For the Ising model, the order 
parameter is the magnetization, which measures the degree of ordering of the spins. 
For a fluid, the order parameter is given by the discontinuity of the density. In both 
cases, the order parameter vanishes above the critical temperature, T^. 

To indicate the generality of the theory, we will denote the order parameter by 
rp. Landau postulated that the free energy F should be an analytic function of ip, as 
indeed it is for both the vdW fluid and the MFA model of ferromagnetism. Since we 
are interested in the behavior near the critical temperature, we expect ip to be small, 
so Landau expanded the free energy in powers of ip. 

F = hip + rip"^ + uip'^ + • • • (30.73) 

In eq. (30.73), the first term indicates the interaction with an external magnetic 
field, which is why h multiplies the lowest odd power of ip. There will, of course, also 
be higher-order odd terms, but we will ignore them in this introduction to the theory. 
For a magnetic model, all odd terms vanish when h = 0 because of the symmetry of 
the Hamiltonian. 

The thermodynamic equilibrium state will be given by value of the order parameter 
that corresponds to the minimum of the free energy. This leads us to assume that 
M > 0, because otherwise the minimum free energy would be found at ip = ±oo. With 
this assumption we can find the minimum by the usual procedure of setting the first 
derivative equal to zero. Ignoring higher-order terms, we find 

OF 

= h + 2rip + 3uip^ = 0 (30.74) 

dip 

First look at the solutions of eq. (30.74) for h = 0. There are two solutions. The 
simplest is ')/’ = 0, which is what we expect for T > Tc. The other solution is 

V, = (30.75) 

Clearly, there is only a real solution to eq. (30.74) for r < 0. This led Landau to assume 
that r was related to the temperature, and to lowest order, 

r = ro{T- T,) (30.76) 


Eq. (30.75) now becomes 


2r„ (Te - T) 

3u 


Ip = zL 


(30.77) 
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Since the definition of the critical exponent (3 in Landau theory is that for very 
small Ip, 


tPot{T,- Tf 


(30.78) 


we see that quite generally, (3 = 1/2. 

All other critical exponents can also be calculated within Landau theory, and their 
values are identical to those obtain from MFA or the vdW fluid. The derivations will 
be left to the reader. 

The calculation of the critical exponents is remarkable for two reasons. The first is 
that the same set of values are obtained from Landau theory for all models. This result 
is known as ‘universality’. It is a very powerful statement. It says that the particular 
interactions in any model of interest are completely unimportant in determining the 
power laws governing critical behavior. The range, strength, or functional form of the 
interactions make no difference at all. 

With some important limitations, universality is even true! 

The second important reason for the importance of Landau theory is that the 
particular values he found for the critical exponents are wrong! They disagree with 
both experiment and some exact solutions of special models. 

The discovery that the Landau theory is wrong was a disaster. A great strength of 
Landau theory is its generality; tweaking parameters has no effect on the values of the 
critical exponents. Unfortunately, that means that if those values are wrong, there is 
no easy way to modify the theory to obtain the right answer. 


30.10 Beyond Landau Theory 

The resolution of the crisis created by the disagreement of Landau theory with 
experiment was not achieved until 1971, when Kenneth Wilson (American physicist, 
1936-) applied renormalization-group theory to the problem. The key to the problem 
turned out to be that at a critical point, fluctuations at all wavelengths interact with 
each other; approximating the behavior of the system without including all interactions 
between different length scales always leads to the (incorrect) Landau values of the 
critical exponents. 

The story of how physicists came to understand phase transitions is fascinating, 
but unfortunately would take us beyond the scope of this book. On the other hand, 
the intention of this book is only to be a starting point; it is by no means a complete 
survey. 

We come to the end of our introduction to thermal physics, which provides the 
foundations for the broad field of condensed matter physics. It is my hope that you are 
now well prepared for further study of this field, which is both intellectually challenging 
and essential to modern technology. 
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30.11 Problems 


Problem 30.1 
An Ising chain 

We have solved the one-dimensional Ising chain with free boundary conditions for 
either J = 0 or h = 0. We also solved the one-dimensional Ising chain with periodic 
boundary conditions with both J ^ Q and h ^ Q. 

For this assignment we will return to the one-dimensional Ising chain with J 0, but 
= 0 . 

1. Use the same matrix methods we used in class to solve the general one¬ 
dimensional chain to find the free energy with J yf 0, but h = 0. (Do not simply 
copy the solution and set h equal to zero! Repeat the derivation.) 

2. Find the free energy per site Fjy/N in the limit of an infinite system for both free 
boundary conditions and periodic boundary conditions. Compare the results. 

3. The following problem is rather challenging, but it is worth the effort. 

A curious problem related to these calculations caused some controversy in the 
late 1990s. Consider the following facts. 

(a) Free boundary conditions. 

The lowest energy level above the ground state corresponds to having all spins 
equal to -1-1 to the left of some point and all spins equal to —1 to the right. 
Or vice versa. The energy of these states is clearly 2 J, so that the thermal 
probability of the lowest state should be proportional to exp(—2/3J), and the 
lowest-order term in an expansion of the free energy should be proportional 
to this factor. 

(b) Periodic boundary conditions. 

The ground state still has all spins equal to 1 ( por —1), but the deviation 
from perfect ordering must now occur in two places. One of them will look 
like 


•••+-k + H- 


and the other will look like 


The energy of these states is clearly 4J, so the thermal probability of 
the lowest state will be exp(—4/3J) for all N. In this case, the lowest- 
order term in an expansion of the free energy should be proportional to 
exp(—4/3J). 

In the limit of an infinite system, this difference is not seen in the free energies 
per site. Can you find the source of this apparent contradiction? 
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Problem 30.2 

Mean Field Approximation (MFA) for the Ising model 

We derived the basic MFA equation for the magnetization in class. 

m = (aj) = tanh(/3z Jto + j3h) 

z is the number of nearest neighbors. For a two-dimensional square lattice, z = 4. 

We found the critical temperature to be given by 

ksTc = zJ 

1. Consider the case of zero magnetic field {h = 0). For temperatures below the 
critical temperature, the magnetization is non-zero. As T Tg, the magnetiza¬ 
tion goes to zero as a power law 

m cx (Tc - Tf 

Find the value of j3. 

2. For temperatures above the critical temperature, find an exact solution for the 
magnetic susceptibility 

dm 

at zero field. 

The magnetic susceptibility diverges at the critical temperature with an expo¬ 
nent called 7 . 

What is the mean-field value of 7 ? 

Suggestions for writing programs: 

The next few problems deal with Monte Carlo simulations of the two-dimensional Ising 
model. In writing the programs, use a sequential selection of spins for the MC steps. 
Even though it is not exactly correct, the differences are very small and a sequential 
selection is much faster. You can implement the algorithm any way you like, but I 
recommend using a loop of the form: 

while n < NJVIC: 

nxm = L — 2 

nx = L — 1 

nxp = 0 
while nxp < L: 
nym = L — 2 

ny = L - 1 

nyp = 0 

while nyp < L: 
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STATEMENTS INSERTED HERE FOR MC UPDATE 


nym = ny 
ny = nyp 
nyp += 1 


nxm = nx 
nx = nxp 
nxp += 1 


STATEMENTS TO RECORD MC DATA INSERTED HERE 
n += 1 

Problem 30.3 

Monte Carlo simulation of a d = 2 Ising model 

Write a computer program to perform a Monte Carlo computer simulation of the 
two-dimensional Ising model in zero magnetic field. 



(ij) 


where the sum is over nearest-neighbor pairs of sites on a square lattice. 

1. Write a program to perform a Monte Carlo simulation of the two-dimensional 
Ising model. You should be able to simulate any size L x L lattice, with L 
determined at the beginning of the program. You should be able to set the 
temperature J at the beginning of the program. You can also write the program 
for zero magnetic field. (We will use non-zero fields in later problems.) 

2. Beginning with a random configuration on a fairly large lattice (L = 32), 
perform a single run of 20 MC sweeps at T = 5.0. and look at the printout 
of the configuration. 

3. Beginning with a random configuration on a fairly large lattice (L = 32), 
perform a single run of 20 MC sweeps at T = 3.0. and look at the print-out 
of the configuration. 

4. Beginning with a random configuration on a fairly large lattice (L = 32), 
perform a single run of 20 MC sweeps at T = 2.0, and look at the print-out 
of the configuration. 

5. Compare the pictures of the configurations that you have generated at T = 
5.0,2.0,2.0. What do you see? 

6 . Beginning with a random configuration on a fairly large lattice (L = 16 or L = 
32), perform several runs with about 50 or 100 MC sweeps at T = 0.5. Keep 
the length of the runs short enough so that you can do it several times without 
wasting your time. Look at the print-outs of several configurations. If you see 
something that seems unusual, print it out and hand it in. Explain why you 
think it unusual. 


388 Phase Transitions and the Ising Model 


Problem 30.4 

Monte Carlo simulation of a d = 2 Ising model (revisited) 

For this problem we will modify the MC program to make a series of runs with 
temperatures separated by a value AT. 

1. Use the Python function definition feature to create a function that does 
the entire MC simulation. Then put the function call inside a loop that first 
equilibrates the spins for some number of sweeps and then generates the data you 
want from a longer number of sweeps. The loop should change the temperature 
by an amount AT for every iteration of equilibration and data-taking. 

2. Calculate the average energy, specific heat, magnetization, and magnetic sus- 
ceptabililty for temperatures of T = 2.0,4.0, and 6.0 for an 8 x 8 lattice. Use a 
large enough numbers of MC updates to obtain reasonable results, but not so 
many that you waste your time sitting in front of a blank computer screen. 

3. What do you notice about the data from your MC simulation? 

This is a fairly open-ended question. The main purpose is to think about the 
results of the simulation. 


Problem 30.5 

More Monte Carlo simulations of a two-dimensional Ising model 

The Hamiltonian of a two-dimensional Ising model is: Use periodic boundary condi¬ 
tions. 


— ij (7j.CTj h fTj 

<i,3> 3 

For this entire assignment, take h = 0. 

Use or modify the program you wrote for an earlier problem to do the following 
simulations. 

1. Simulate a 16 x 16 lattice with J = 1.0. Begin with giving each spin a random 
value of -1-1 or —1. Scan the temperature from 5.0 to 0.5 at intervals of AT = 
—0.5. Use about 100 or 1000 MC sweeps for each temperature, depending on 
the speed of your computer. 

Make a table of the results and plots of m vs. T, E vs. T, c vs. T, and x 
vs. T. 

Does it look as if the critical temperature is at the mean field value of ksTc = 
4J? 

2. Repeat the sweeps, but this time choose the temperature range to concentrate 
on the peaks in the specific heat and the magnetic susceptibility. Estimate the 
location and height of the peak in specific heat and magnetic susceptibility. 
These will be estimates of the critical temperature. 

3. And now for something completely different. 
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Do a simulation of a 32 x 32 lattice with negative temperature, and print out 
the spins at the end of the run. Begin with the spins in a random state {T = oo), 
and simulate for 100 sweeps at T = —1.0. 

What do you see? 

Problem 30.6 

A one-dimensional, spin-one model of magnetism 

Consider a one-dimensional macroscopic model with spin-one quantum mechanical 
magnetic moments located on each of N sites. The Hamiltonian is 

Af-l N N 

H = -jJ2 

i=i i=i 

where each aj takes on the values —1, 0, or -|-1, J is an exchange interaction parameter, 
is a magnetic field, and D is a parameter representing a ‘crystal field’. The entire 
system is in contact with a thermal reservoir at temperature T. 

1. Write down a formal expression for the partition function of this system. You 
do not have to evaluate any sums or integrals that appear in your expression 
for this part of the problem. 

2. For general values of the parameters, derive a formal expression for the quantity 

in terms of a derivative of the partition function with respect to an appropriate 
parameter. 

3. For J = 0, but arbitrary values of h and D, calculate the partition function for 
this system. 

4. For J = 0, but arbitrary values of h and D, calculate the free energy of this 
system. 

5. For J = 0 and h = 0, calculate the entropy of this system. 


Appendix: 

Computer Calculations and VPython 


There are a number of problems in this book that require computer calculations. These 
problems are not supplementary material; they are an essential part of the book. They 
are designed to provide a deeper insight into the physics of many-particle systems than 
would be possible through analytic calculations alone. This Appendix provides sample 
programs to support your own programming. 

There are definite advantages to writing computer programs yourself. Writing a 
program requires focussing on details of the mathematical structure of a problem and, 
I hope, understand the physics more deeply. 

The choice of programming language is up to you. Over the life of this book, 
programming languages will certainly change. However, the basic ideas of computer 
programming will continue to be useful and applicable to new improvements and 
developments. 

The examples presented here use the VPython programming language, which I 
strongly recommend to newcomers to computation. It runs on any computer using 
any operating system. It is easy to learn, even if you’ve never programmed before. 
And the price is right: you can download it for free! 

I’ve provided some information about VPython programs to help you get started. 
The first program I’ve given you is almost—but not quite—sufficient to carry out the 
first assignment. However, if you know what every command means and what you 
want the program to do, the necessary modifications should be fairly straightforward. 
If there is anything about programming you do not understand, do not be shy about 
asking the instructor. 

A.l Histograms 

Since an important part of the first program is a print-out of a histogram, it is 
important to understand how histograms work. 

A histogram simply records a count of the number of times some event occurs. In 
our first program, ‘OneDie.py’ (given in the next section), an event is the occurrence of 
a randomly generated integer between ‘0’ and ‘5’. In the program, the number of times 
the integer n occurs is recorded in the memory location ^histogram[n]’. For example, 
if the number ‘4’ occurs a total of 12 times during the course of the simulation, then 
the value stored in histogram.[4\ should be 12. The command 

print ( histogram [4] ) 

should contain the number ‘12’. 
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Histograms can be represented in various ways in a VPython program. I have 
chosen to represent them by an array. The command 

histogram = zeros (sides , int ) 

creates an array named ‘histogram’ with ‘sides=6’ memory locations, and sets the 
value stored in every memory location equal to zero. 

The command 

histogram [r] = histogram [r] +1 
increments the number stored in the r-th location. It could also be written as 
histogram [r] += 1 

to produce the same effect. Commands of the form +=, -=, *=, and /= probably 
seem strange the first time you see them, but they are very useful. 

It is quite common to confuse a histogram with a list of numbers generated by the 
program. They are completely different concepts. The length of the histogram is 
given by the number of possibilities—six in the case of a single die. The length of 
the list of random numbers is stored in the memory location ‘trials’; it can easily 
be of the order of a million. Printing out a histogram should not take very long. 
Printing out a list of random numbers can exceed both your paper supply and your 
patience, without producing anything useful. When writing a program, be careful 
that you do not make this error. 


A.2 The First VPython Program 

The following is a sample VPython program to simulate rolling a single die. Note that 
either single quotes or double quotes will work in VPython, but they must match. 
Double single quotes or back quotes will give error messages. 

from visual import * 

from visual, graph import * 

import random 

import sys 

from types import * 

from time import clock , time 

trials = 100 

print ( ” ^Number^of ^ t ri als , trials) 
sides = 6 

histogram = zeros ( sides , int ) 
print ( histogram ) 
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sum = 0.0 

j=0 

r=0 

while j < trials: 

r = int (random . random () * sides ) 
histogram [r] = histogram [r] + 1 

j=j+l 

j=0 

while j < sides : 

print ( histogram [ j ] , histogram[j] — t r i als / sides ) 

j=j+l 

The first six lines in the program given above are called the ‘header’ and should 
simply be copied at the beginning of any program you write. Blank lines are ignored, so 
for clarity they can be put them in anywhere you like. Indentations are meaningful: in 
the while command, every indented command will be carried out the required number 
of times before going on the next unindented command. While typing in the program, 
you will notice that VIdle indents automatically. 

The function 

random . random () 

produces a (pseudo-)random number between 0.0 and 1.0. Multiplying it by 6 produces 
a random number between 0.0 and 6.0. The command 

r = int (random . random () * sides ) 

then truncates the random number to the next lowest integer and stores the result in 
r, so that r is an integer between 0 and 5. 

The program given above should run exactly as written. Copy the program to your 
computer and try it! If it does not work, check very carefully for typographical errors. 

Warning: Computers have no imagination. They do exactly what the program tells 
them to do—which is not necessarily what you want them to do. Sometimes a 
computer can recognize an error and tell you about it; but life being what it is, the 
computer will probably not recognize the particular error that is preventing your 
program from running properly. This might be another consequence of the Second 
Law of Thermodynamics, as discussed in Chapter 11. 


Make a permanent copy of the program given above, and then make your modifi¬ 
cations in a new file. This is a good policy in general; base new programs on old ones, 
but do not modify a working program unless you have a back-up copy. 
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Experiment with making modifications. You cannot break the computer, so do not 
worry about it. If the program just keeps running after a few seconds, you can always 
stop it by closing the Python shell window. 

If there is anything about the program that you do not understand, ask someone. 
If you understand everything in this program, the computational work for all of the 
rest of the problems will be much easier. 

A.3 VPython Functions 

The Python language allows you to define functions in a very natural way. They are 
really quite easy to write, and can simplify your program enormously. 

The following code is an example for defining a function to calculate the energy of 
a simple harmonic oscillator: 

def ENERGY( x, K ): 

energy = 0.5 * K * x * x 
return energy 

The energy E can then be calculated for any value of position x and spring constant 
K by the command: 

E = ENERGY( x, K ) 

One peculiarity of Python is that you must define a function before you use it in 
the program. 

There is another interesting feature of Python (and VPython) that can affect how 
you use functions. If you are using ‘global’ variables—that is, variables defined in 
the main program —and not changing their values in the program, you do not have to 
include them in the list of arguments. For example, the code above could be written as: 

def ENERGY( ) : 

energy = 0.5 * K * x * x 
return energy 

This would give the same answer as before, as long as both ‘K’ and ‘x’ had been 
defined before ‘ENERGY()’ was called. 

However, be careful if you want to change a global variable in a program. As 
soon as you write ‘x=...’, ‘x’ becomes a local variable, and VPython forgets the global 
value. This can be a source of much annoyance. If you want to change a global variable 
inside a function, include a statement such as ‘global x’ in the function for each global 
variable. 

Curiously enough, if you are using a global array, you can change its values in a 
function without declaring it to be global. I am sure that there is a very good reason 
for this discrepancy, but I do not have any idea of what it might be. 

A.4 Graphs 

VPython has very nice programs to produce graphs. However, they can be rather 
daunting the first time you use them. The following VPython program provides 
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examples for plotting multiple functions in a single graph. Three different ways to plot 
functions are demonstrated. A second window is also opened at a different location 
on the screen with yet another function. 

from visual import * 

from visual, graph import * 

import random 

import sys 

from types import * 

from time import clock , time 

^ Program, to demonstrate various plotting options 

MAX = 30 

^ The next command opens a plotting window 

graphl = gdisplay (x=0, y=0, width=400, height=400, 
t i 11 e= ’ Trig „functions ’ , 
xtitle =’x„’, 
ytitle =’y.’ , 

#xmax = 1.0 * MAX , 

#xmin = 0.0, 

^ymayi — 100.0 , 

#ymin ^0.0, 

foreground=color . black , background=color . white) 

k=l 

Apoints = [] 
while k < MAX: 

Apoints += [(k * pi / MAX, cos ( k * pi / MAX ) )] 
k=k+l 

datal = gvbars ( delta=2./MAX, color=color . red ) 
datal . plot ( pos=Apoints) 


k=l 

Bpoints = [] 
while k < MAX: 

Bpoints += [(k * pi / MAX, —sin ( k * pi / MAX ) )] 

k=k+l 

data2 = gdots(color=color.green) 
data2 . plot ( pos = Bpoints) 
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k=l 

Cpoints = [] 
while k < MAX: 

Cpoints += [(k * pi / MAX, sin( k * pi / MAX ))] 
k=k+l 

dataS = gcurve(color=color.blue) 
dataS . plot (pos = Cpoints) 

^ The next command opens a second plotting window 

graph2 = gdisplay (x=401, y=0, width=400, height=400, 
t i t le = ’Second „plot ’ , 

Xt i t le= ’X^ ’ , 
ytitle = ’Y^ ’ , 
xmax =2.0 , 

#xmin = 0.0, 

:?^ymax = 1.10, 

#ymin ^0.0, 

foreground=color . black , background=color . white) 


k =0 

Dpoints = [] 
while k < MAX: 

Dpoints += [(2.0 * k / MAX, exp ( — 2.0 * k / MAX ) )] 
k=k+l 

data4 = gcurve(color=color.blue) 
data4 . plot ( pos=Dpoints ) 

A.5 Reporting VPython Results 

Results of computational problems, including numbers, graphs, program listing, and 
comments, should be printed out neatly. Here are some hints for doing it more easily 
in 

A.5.1 Listing a Program 

IMdcX has a special environment that was used to produce the program listings in this 
Appendix. To use it, you need to include the following statement in the header: 

\usepackage{listings} 

\lstloadlanguages{Python} 

\lstset-[language=Python, coininentstyle=\scriptsize} 

The command for the listing environment is: 
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\begin{lstlisting} 

Program goes here 
\end{lstlisting} 


A.5.2 Printing—Pretty and Otherwise 

The printed output of a VPython program is always found in the Python shell window. 
The simplest procedure is just to copy it into your favorite word processor. However, 
you might want to clean up the output of your VPython program first by specifying the 
width of a column and the number of significant digits printed. Python and VPython 
have formatting commands that are inserted into a ‘print-statement. I think the 
commands are most easily explained by giving an example. 

Example: Setting Column Width 

As an illustration of the default output, consider the following statements: 

print ( ’’spin”, ’’hist”, ” hist—average ” , ”freq”, ” freq—1/sides ” ) 

and, indented inside a loop, 

print ( j, h, hDiff , hOverTrials , hOSdiff) 
produce the following output: 

spin hist hist—average freq freq—1/sides 
2 0.333333333333 0.2 0.0333333333333 

The more highly formatted statement (on multiple lines in this example, with 
commas indicating the continuations) 

print ( ’%.6s’ % ’’spin”, ’%.7s ’ % ’’hist”, 

’%.T0s ’ %” deviation” , ’%.8s ’ %”freq” , 

’%-T2s’ %” deviation”) 

print( ’%.6d’ % j , ’%.6.2f ’ % h, ’%. 10.4 f ’ % hDiff , 

’%.8.4f’ %h OverTrials , ’%.12.4f’ % hOSd) 

produce the following output: 

spin hist deviation freq deviation 

0 165.00 -1.3333 0.1650 -0.0017 

In the expression ‘6s’, the ‘6’ refers to the column width, and ‘s’ means that a string 
(letters) is to be printed. In the expression ‘6d’, the ‘6’ again refers to the column 
width, and ‘d’ means that an integer is to be printed. In the expression ‘10.4f’, the 
‘10’ refers to the column width, the ‘.4’ refers to the number of digits after the decimal 
point, and ‘f’ means that a floating point number is to be printed. 
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A.5.3 Printing Graphs in LaTeX 

Under any operating system, there are efficient utility programs to capture an image 
of the screen, a window, or a selected area. Use them to make a file containing the 
output of the graphics window and copy it into your word processor. If you are using 
as I am, you can make a pdf file and then put it into your file as follows: 

\begin{figure}[htb] 

\begin{center} 

\includegraphics[width=4in]{name_of_pdf_file} 

\caption{Caption goes here} 

\label{your choice of label here} 

\end{center} 

\end{figure} 

You will also need to include 
\usepackage{graphicx} 
in your header. 

A.6 Timing Your Program 

If your program includes the header command 
from time import clock , time 

timing your program, or parts of your program, is easy. Placing the command 
to = clock() 

at the beginning of your program and something like 

ProgramTime = clock () — tO 

print ( ” „Program„time, ProgramTime ) 

at the end of the program, produces a printout of how long the program took to run. 
This information can be very useful in checking the efficiency of you program and 
planning how many iterations you should use in doing the assignments. 

There is also a function ‘time()’ that can be used in the same way as ‘clock()’. 
The difference is that ‘clock()’ will calculate how much CPU time was used by your 
program, while ‘time()’ will calculate how much real time has passed. Both are useful, 
but ‘clockO’ gives a better measure of the efficiency of the program. 

A.7 Molecular Dynamics 

The key loop in a molecular dynamics (MD) program is illustrated below. The function 
‘FORCE’ is assumed to have been defined earlier, along with the parameter K. 
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^ the velocity is assumed to have been initialized 
^ before the beginning of the while—loop 

n=0 

while n < NJVID: 
n += 1 

X += dt * velocity 

velocity += dt * FORCE( x, K) 

The algorithm given above is suggested to simplify the programming. A better 
iteration method is the ‘leap-frog’ algorithm, in which the positions are updated at 
even multiples of St/2, while velocities are updated at odd multiples. If you are feeling 
ambitious, try it out. You can check on the improvement by looking at the deviation 
from energy conservation. 

A.8 Courage 

The most important thing you need to run computer programs is courage. For example, 
when you are asked to run the first program for different numbers of trials, do not be 
afraid of using a large number of trials. Do not just use five trials vs. eight trials. Start 
with a small number such as 10, but then try 100, 1000, or higher. As long as the 
computer produces the answer quickly, keep trying longer simulations. However, if the 
computer takes more than about one second to finish, do not use longer simulations— 
your time is also valuable. 
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